Note EFSPI 9th RegStat Workshop - Part A

1 EFSPI regulatory statistics workshop

1.1 Introduction

1. EFSPI Meaning: EFSPI stands for the European Federation of Statisticians in the Pharmaceutical Industry. It’s a collective body representing statisticians involved in the pharmaceutical sector across Europe.

2. Foundation Year: EFSPI was founded in 1992, marking its establishment as an entity to organize and represent statisticians within the pharmaceutical industry at a European level.

3. Organizational Structure: EFSPI is described as an “umbrella” organization, indicating that it functions at a higher organizational level that encompasses various national groups rather than individual members. It is a non-profit organization, which means it does not operate to generate profit but rather to serve the interests of its members and the profession.

4. Federation Composition: The federation consists of national groups from different European countries, currently totaling 10. These groups collectively represent the interests of statisticians in the pharmaceutical industry at a national level.

5. Membership: It is noted that EFSPI does not have individual members. Instead, its structure is based on national organizations which collectively represent over 2000 members. This large number suggests a broad and significant representation within the pharmaceutical statistics community in Europe.

6. Representation on the EFSPI Council: Each national organization that is part of EFSPI has two members representing it on the EFSPI Council. This council likely serves as the governing body or decision-making panel for the federation.

7. Website: For more information, reference to their activities, publications, events, or any specific details about the federation’s operations and initiatives, the website provided is www.efspi.org.

1.2 Objectives

1. Promote Professional Standards of Statistics:
   • EFSPI aims to elevate and maintain high professional standards in the application of statistics within the pharmaceutical industry. This involves advocating for integrity, reliability, and excellence in statistical practices, which are crucial in research and development processes in the industry.
2. Offer Collective Expert Input on Statistical Matters:
   • The federation serves as a bridge between statisticians in the pharmaceutical industry and national and international regulatory authorities. By providing collective expert opinions and input on statistical matters, EFSPI influences policy-making and regulatory standards, ensuring that statistical methodologies meet the required guidelines and are appropriately implemented within the industry.
3. Exchange Information and Harmonise Attitudes to the Practice of Statistics:
   • EFSPI facilitates the sharing of information and best practices among its members and between member groups. This objective is geared towards harmonizing methodologies and attitudes towards statistical practice across different regions and companies within the European pharmaceutical sector. This collaborative approach helps in standardizing procedures and improving the overall quality of statistical work within the industry.

1.3 10th ESFPI Regulatory Statistics Workshop

• 10-12 Sep 2025
• Basel Switzerland

2 Session 1: Fast to market vs. robustness of the data

Chairs: Khadija Rantell (MHRA, UK) and Fredrik Öhrn (J&J, SE)

2.1 S1-Talk 1: How the pressure to be first, faster, puts pressure on us all and what we can do about it?

Speaker: Jenny Devenport (Roche, CH)

Robustness and Uncertainty

Robustness - Definition: In statistics, robustness refers to the ability of statistical estimates to remain consistent and reliable even when there are violations of the assumptions upon which these estimates were initially based. - Context: This property is crucial because in real-world data analysis, assumptions such as normal distribution, homogeneity of variance, or linearity often do not hold perfectly. Robust statistical methods can still provide valid results despite such issues as non-normality, heteroscedasticity (non-constant variance), or outliers in the data. - Examples: An example of a robust statistical method is the median as a measure of central tendency. Unlike the mean, the median is not unduly affected by extreme values (outliers).

Uncertainty

• Definition: Uncertainty in statistics refers to the inherent variability observed in the data and the estimates derived from statistical analysis.
• Types of Uncertainty:
  • Sampling variability: This arises because a sample is only a part of the entire population, and different samples might yield different estimates.
  • Measurement error: This type of uncertainty occurs when there is error in how data is measured or recorded, which can affect the accuracy of the resulting data.
• Importance: Recognizing and quantifying uncertainty is essential for making informed decisions based on statistical analysis. It is often expressed through confidence intervals or hypothesis tests.

Challenges in Drug Development

Tthree main drivers: 1. Patients are Waiting: There’s a pressing need from patients who are in urgent need of new treatments, which creates a moral and social imperative to speed up the development and availability of drugs. 2. Healthcare Systems Need Effective Treatments: Efficient treatments are essential to alleviate the burden of disease on healthcare systems, which can improve patient outcomes and reduce overall healthcare costs. 3. Sustainable Innovation in the Pharmaceutical Industry: Innovation needs to be sustainable, ensuring that new developments are both effective and financially viable over the long term. This includes not only creating new drugs but also ensuring they can be produced and distributed sustainably.

Patient Pressure to Increase the Speed of Drug Development

This slide details specific ways in which patient advocacy groups are influencing faster drug development:

1. Compassionate Use Programs: These programs allow patients to access new drugs before they receive formal regulatory approval, usually in cases where no other treatment options are available.
2. Engagement in Drug Development:
   • Increasing Therapies: Working with multiple sponsors to boost the number of therapies under development.
   • Improving Clinical Trials: Engaging patients to enhance enrollment timelines and retention rates in clinical trials.
   • Endpoint Selection: Influencing the selection of patient-relevant endpoints, which can lead to more meaningful and impactful clinical trial outcomes.
   • Regulatory Testimony: Patients and advocates testifying at regulatory or advisory committee meetings to ensure that patient perspectives are considered.
   • Post-Marketing Safety: Advocating for more effective safety monitoring after a drug has been marketed, ensuring that drugs continue to be safe for the public.

Both together emphasize a multi-faceted approach to accelerating drug development, involving patients directly and addressing systemic needs for efficiency and sustainability in healthcare. These efforts aim to ensure that new treatments not only reach the market more quickly but do so in a way that is safe and addresses the real needs of patients.

Challenges in Modern Drug Development

• Quality of New Medications: There are already many effective medicines available, so new drugs must provide substantial improvements or benefits to stand out. This is humorously referred to as needing to be “Better than the Beatles,” implying that new drugs must surpass high existing standards.
• Higher Sample Sizes: New treatments often require larger clinical trials to demonstrate incremental improvements over existing therapies, making the development process more extensive and expensive.
• Complex Programs: To differentiate a new drug’s performance effectively, more complex and comprehensive development programs are required, further adding to the cost and complexity of bringing a drug to market.
• Market Differentiation: Without significant improvements, new drugs may face challenges in market penetration and pricing. Drugs that do not significantly outperform existing options might have to compete by lowering prices, which can impact profitability.

Implications

• Economic Pressure: The increasing cost and complexity of drug development can lead to economic pressures on pharmaceutical companies, potentially limiting innovation.
• Regulatory and Market Challenges: Companies must navigate rigorous regulatory requirements and market expectations that demand clear and substantial benefits of new drugs over existing therapies.

Eroom’s Law

The slide you’ve shared addresses a significant trend in pharmaceutical research and development (R&D) efficiency, encapsulated by what’s known as “Eroom’s Law.” Here’s a breakdown and explanation of the concepts and issues presented:

Eroom’s Law - Definition: Eroom’s Law is the observation that drug discovery is becoming slower and more expensive over time, despite improvements in technology and science. It’s actually “Moore’s Law” spelled backwards, referring to the opposite trend observed in the electronics industry, where technology becomes faster and cheaper. - Trend Explanation: The graph shows a decline in R&D efficiency, which is measured as the number of new drug approvals per billion U.S. dollars spent. This decline suggests that despite increased investment in drug development, the output in terms of successful new drugs has not proportionally increased.

Patient Pressure

• Advocacy and Engagement: Patient groups are increasingly involved in drug development, influencing various stages from clinical trial design to post-marketing safety evaluations. This involvement helps ensure that new treatments address actual patient needs and safety concerns effectively.
• Regulatory Testimony: Patients are actively participating in regulatory advisory committee meetings, advocating for faster and more responsive drug approval processes.

Healthcare System Pressure

• Quick Response Needed: The COVID-19 pandemic exemplified how healthcare systems can mobilize and accelerate drug development when there is a unified effort and clear, immediate need.
• Financial Protection: There’s an emphasis on protecting individuals from the financial burdens of healthcare, particularly when treatments are expensive. Effective drugs can reduce long-term healthcare costs by improving health outcomes.

Industry Pressure

• Financial Incentives: The industry faces its own set of pressures, primarily financial. Being first to market can lead to substantial market share and long-term revenue advantages, as demonstrated by drugs that remain market leaders years after their release.
• First-to-Market Advantage: Drugs that are first in their class often secure a significant competitive edge, which can be sustained through additional indications or superior performance.

Statistician’s Role

• Challenges in Demonstrating Improvement: There’s a growing need for larger sample sizes and more complex clinical programs to demonstrate incremental improvements over existing treatments. This requirement stems from high existing standards and the necessity to clearly differentiate new products in the market.
• Quality Over Speed: The focus is on the quality of the data and the direction of the research. Speeding up drug development without maintaining rigorous statistical and clinical standards could lead to inefficiencies or worse, ineffective or unsafe medications.

How statisticians can contribute to drug development

Statisticians provide a framework for addressing complex questions that drug development entails. Their expertise in quantitative analysis, experimental design, and critical evaluation of data ensures that pharmaceutical companies can navigate the intricate process of bringing a new drug to market effectively. They play a pivotal role in ensuring that the drugs that reach the market are backed by robust evidence, ultimately influencing healthcare outcomes positively. This role is crucial in maintaining the integrity of the drug development process, especially under the pressures of speed and innovation that the industry faces.

1. Structuring the Problem / Asking Good Questions

• Role: Statisticians help define and structure research questions and hypotheses clearly and rigorously.
• Impact: By asking the right questions, they ensure that the research addresses specific, meaningful, and scientifically valid points that are crucial for successful drug development.

Details

• Specific Question: Each step in drug development should target a specific question, making the goals clear and actionable.
• Efficiency Opportunities: By recognizing when existing data already provides answers, opportunities for efficiency can be identified, potentially speeding up the process.
• Clarity and Assumptions: Being clear about the research questions and transparent about the assumptions being made are crucial for generating the right data.
• Value of ‘No’: Understanding that a ‘no’ answer is still valuable helps maintain objectivity and prevents unnecessary progression on ineffective paths.

2. Making a Series of Good Decisions

• Role: Statisticians use data to inform decisions throughout the drug development process. This involves choosing the right statistical methods, deciding on study designs, and determining what data to collect.
• Impact: Good decision-making based on sound statistical principles can significantly affect the efficiency of the development process and the reliability of the results.

3. Designing & Running Appropriate Trials to Answer Questions

• Role: Statisticians are integral in designing clinical trials that are capable of providing clear, unbiased, and interpretable results. This includes determining sample sizes, selecting control groups, and defining endpoints.
• Impact: Properly designed trials are essential to ensure that the drug is effective and safe, guiding regulatory approval and market acceptance.

4. Interpreting Evidence Properly

• Role: Statisticians interpret the data from trials to draw conclusions about the efficacy and safety of a drug.
• Impact: Accurate interpretation prevents incorrect conclusions from being drawn, which can misguide further development, regulatory decisions, or clinical practice. Proper interpretation also involves understanding the limitations of the data, including recognizing biases, variability, and uncertainty.

Risk Mitigation in Accelerated Programs

• Futility Analyses: Implementing futility analyses with meaningful boundaries ensures that resources are not wasted on likely unsuccessful paths.
• Adaptive Designs with Criteria: These designs allow for trial adjustments based on predefined criteria, managing risks dynamically throughout the trial.
• DSMBs / IDMCs: Data and Safety Monitoring Boards or Independent Data Monitoring Committees play critical roles in overseeing trial progress and safety, providing an external check on study conduct.
• Collaboration with Authorities: Maintaining close collaboration with health authorities and other stakeholders helps align the drug development process with regulatory expectations and best practices.

Tools and Strategies for Accelerated Development

• Regulatory Pathways: Various accelerated assessment and approval mechanisms are available across jurisdictions, which can facilitate quicker market entry for new drugs. Understanding and navigating these pathways effectively can significantly impact the speed of development.
• Advanced Statistical Methods: Tools like adaptive designs, patient enrichment strategies, and surrogate endpoints can help streamline the development process. These methods allow for more dynamic adjustments during trials, improving the efficiency and potentially leading to earlier conclusions about a drug’s efficacy and safety.

Details

• Beyond Statistics: Statisticians are encouraged to think like drug developers, focusing on broader project goals rather than just statistical metrics.
• Adaptive Designs: Using adaptive designs allows for modifications to the trial or study based on interim results, potentially saving time and resources.
• Platform Trials: Testing multiple products or combinations in a single trial can streamline processes and enhance operational and statistical efficiency.
• Sample Size and Precision: Strategies such as using surrogate endpoints, patient enrichment, and covariate adjustments can reduce required sample sizes or improve the precision of estimates.

Integration of AI and ML in Drug Development - Technological Advances: Despite significant technological advancements in areas like combinatorial chemistry, DNA sequencing, and high-throughput screening, the expected surge in drug approvals hasn’t materialized. This discrepancy highlights the complex nature of translating scientific advances into practical, marketable therapies. - Role of AI and ML: These technologies are seen as potential game-changers in drug development. However, their effective integration requires a clear understanding of their capabilities and limitations. Statisticians and drug developers are encouraged to think critically about how AI and ML can be strategically deployed to improve various stages of drug development, from target identification to post-marketing surveillance.

Conclusion

• Speed in Drug Development:
  • The speaker uses “speed” to discuss the time it takes to gather substantial evidence enough to support drug approvals. They question when we have sufficient right data to deem it substantial for regulatory purposes.
• Technological Advances:
  • They plan to discuss technological advancements that could have, but perhaps haven’t fully, accelerated drug approvals.
• Final Thought on Speed: Concludes with a thought-provoking statement that “speed never killed anyone in drug development,” emphasizing that the real danger lies in errors made during the hurried processes, not the speed itself.

2.2 S1-Talk 2: Conditional marketing authorisation.

Speaker: Eva Skovlund (NOMA, NO; CHMP member)

The framework of CMAs with specific obligations underscores the dynamic nature of drug approval and the need for a balance between rapid access to important new therapies and ensuring their continued safety and efficacy through rigorous post-market evaluation.

Conditional Marketing Authorisation (CMA)

The detailed overview you provided highlights the concept of Conditional Marketing Authorization (CMA), a regulatory mechanism designed to facilitate the expedited approval of medications that meet urgent and significant medical needs, particularly in challenging situations like life-threatening diseases, emergencies, or when no other treatments are available. Here’s an expanded explanation of the scope and criteria for CMA:

Scope of Conditional Marketing Authorization

1. For Seriously Debilitating or Life-Threatening Diseases: CMAs are often granted for treatments that target severe health conditions where patients have limited options.
2. Emergency Use: This includes situations like pandemics where rapid deployment of treatments is critical to address public health emergencies.
3. Orphan Medicinal Products: These are drugs developed for rare diseases (often referred to as orphan diseases) that typically wouldn’t be profitable to produce without regulatory incentives.

Criteria for Conditional Marketing Authorization

1. Positive Benefit-Risk Balance: The benefits of the drug must clearly outweigh any risks, taking into account the severity and urgency of the need for treatment.
2. Likelihood of Providing Comprehensive Data Post-Authorization: There’s an expectation that more complete and extensive data will be provided after the drug is on the market, to further verify its efficacy and safety.
3. Fulfillment of an Unmet Medical Need: The medication should address a need not currently met by existing treatments, offering new therapeutic benefits to patients.
4. Immediate Need Over Risk: The immediate availability of the medicine provides greater benefits than the risks posed by the fact that additional data are still needed.

Additional Considerations

• Ongoing Data Collection: Post-marketing surveillance and additional studies are typically required to gather further evidence on the drug’s effectiveness and safety.
• Review and Renewal: CMAs are not permanent and require periodic review where the latest collected data are assessed to determine if the terms of the authorization continue to be met.
• Practical Application and Implications The use of CMA has been particularly noted during the COVID-19 pandemic, where treatments and vaccines were urgently needed. This regulatory pathway allowed for the faster deployment of potentially life-saving medicines under conditions where traditional full marketing authorizations would not be feasible or timely.

Intent and Outcomes of CMAs

• Standard of Proof: The intent behind CMAs is not to lower the standards for approval but to balance expedited access to potentially life-saving drugs with the need to confirm their benefits through additional data.
• Positive Outcomes Expected: Generally, there is an optimistic view that most products approved under CMAs will eventually fulfill their SOBs and convert to full marketing authorizations. This expectation is based on the assumption that the additional data collected will support the initial positive benefit-risk assessment.
• Regulatory Consequences: If the studies required by the SOBs fail to confirm the expected outcomes, or if there are significant delays in fulfilling these obligations, the drug’s approval can be reevaluated, which might lead to a revocation or suspension of the marketing authorization.

Implications for Drug Development and Approval

• Access vs. Assurance: CMAs offer a pathway to bring treatments to market quickly, particularly in areas of high medical need. However, this expedited access must be balanced with thorough and timely verification of the drug’s benefits through post-marketing commitments.
• Long-Term Commitment: Companies must be prepared for a long-term commitment to comprehensive data collection and analysis to meet SOBs. This involves careful planning from the outset to ensure that adequate resources and structures are in place to conduct the necessary post-approval studies.

Post Approval Commitments/Specific Obligations (SOB)

1. Obligation Fulfillment: Once a CMA is granted, the Marketing Authorization Holder (MAH) must fulfill specific obligations within defined timelines. These obligations are designed to ensure that additional data are collected to support the continued use of the drug under real-world conditions.

2. Types of Commitments:

   • Completing Ongoing or New Studies: This may involve continuing pre-existing studies or initiating new ones to further assess the drug’s effectiveness and safety.
   • Collecting Additional Data: Gathering further data to confirm that the medicine’s benefit-risk balance remains favorable. This could include additional clinical trials, observational studies, or other forms of data collection.

3. Conversion to Standard Marketing Authorization:

   • If the MAH fulfills these obligations and the complete data affirm that the medicine’s benefits continue to outweigh its risks, the CMA can be converted into a standard marketing authorization.
   • This conversion is crucial as it transitions the drug from a conditionally approved status to full approval, indicating a more robust validation of its clinical benefits and safety.

4. Risks of Non-Compliance:

   • If new data reveal that the benefits of the medicine no longer outweigh its risks, or if the MAH fails to comply with the imposed obligations, the marketing authorization (MA) can be suspended or revoked.
   • This mechanism ensures that the drug remains under scrutiny post-approval and that any emerging risks are adequately managed.

5. Regulatory and Practical Implications:

   • The obligations often involve conducting additional comparative trials or around-trials, which are crucial for gathering comprehensive efficacy and safety data.
   • The timelines for these obligations can vary significantly, sometimes extending over many years, depending on the complexity and nature of the required data.

Importance of Post-Approval Commitments

• Continuous Monitoring: These commitments ensure ongoing monitoring of the drug’s performance in broader and more diverse patient populations than those typically included in pre-approval clinical trials.
• Risk Management: They provide a mechanism to manage potential risks that might not have been fully ascertainable during the initial approval process, safeguarding public health.
• Regulatory Assurance: Fulfilling these obligations is critical for maintaining the integrity of the expedited approval process and ensuring that the benefits of a conditionally approved drug continue to justify its availability on the market.

Challenges with Post-Approval Specific Obligations (SOBs)

• Trial Initiation and Duration: There’s an acknowledgment that initiating and conducting the required post-approval studies can be time-consuming. Delays in starting these trials can significantly push back the timeline for fulfilling SOBs.
• Surrogate vs. Clinical Endpoints: The use of surrogate endpoints in trials, especially when clinical endpoints (which directly measure clinical benefit) are lacking, is a critical point of discussion. Surrogate endpoints can expedite trials but may not always directly correlate with the actual benefits to patients, which complicates the assessment of a drug’s true effectiveness.
• Population Relevance: The trials conducted as part of SOBs may not always involve the same population segments as those initially used for the CMA. This variation can affect the generalizability and relevance of the trial results to the broader patient population.
• Regulatory Expectations: There’s a dialogue around whether trials should already be underway at the time of CMA granting. The FDA and other regulatory bodies might be moving towards expecting more immediate progress post-authorization, which could help ensure that the SOBs are met in a more timely manner.

Challenges

The excerpt you’ve shared delves into complex and nuanced challenges associated with Conditional Marketing Authorizations (CMAs), particularly when dealing with objective response rates as primary endpoints in clinical trials, the difficulties of using real-world evidence, and specific examples where CMAs have been both problematic and instructive. Here’s a deeper exploration and explanation of the key points:

Challenges with Objective Response Rates

• Objective Response Rate (ORR): This measure is often used in oncology trials to assess how well tumors respond to treatment, typically measured by the percentage of patients whose tumor size decreases by a certain amount. However, ORR doesn’t necessarily correlate with longer-term outcomes like overall survival (OS) or progression-free survival (PFS), which are more indicative of clinical benefits to patients.
• Threshold Variability: There is no universal threshold for what constitutes a meaningful ORR. This threshold can vary depending on the disease being treated and the historical context of treatment outcomes within that indication. Determining what ORR is clinically significant is subjective and can lead to variability in how drugs are assessed for approval.
• Statistical Confidence: Concerns about the width of confidence intervals around ORR estimates highlight the statistical uncertainty often present in trials used to support CMAs. Small patient numbers can exacerbate this issue, leading to less reliable estimates of treatment effect.

Use of Real-World Evidence

• Challenges of Real-World Data: While real-world evidence can provide insights into how a drug performs outside the controlled conditions of clinical trials, there are significant challenges and biases associated with this type of data. These include issues with data quality, patient selection biases, and confounding factors, which can complicate the interpretation of real-world evidence.
• Integration in Regulatory Decisions: Real-world evidence is still a contentious and evolving area within regulatory science, particularly in its role in confirming the efficacy and safety of drugs approved under CMAs.

Specific Examples Highlighting CMA Challenges

Let’s delve deeper into the examples you mentioned, focusing on the Conditional Marketing Authorization (CMA) challenges associated with treatments for Multiple Myeloma and Primary Ciliary Dyskinesia. These case studies provide valuable insights into the complexities of drug approval processes and post-marketing commitments.

These examples highlight several key points in the CMA process:

1. Importance of Rigorous Post-Approval Studies: Both examples illustrate why CMAs are often contingent on subsequent studies to validate initial findings. These studies are crucial to ensure that the benefits initially observed are real and meaningful.

2. Challenges with Surrogate Endpoints: Relying on surrogate markers or ORR can lead to approvals based on data that may not fully capture the drug’s impact on disease progression or patient quality of life.

3. Need for Holistic Assessment: These cases underscore the need for a comprehensive assessment that considers long-term outcomes and real-world effectiveness to truly gauge the value of new treatments.

Example of Multiple Myeloma Treatment

Background: Multiple Myeloma is a type of blood cancer that affects plasma cells. Treatments typically aim to manage symptoms and prolong survival through various therapies, including chemotherapy, targeted therapy, and stem cell transplants.

Challenges Faced: - Objective Response Rate (ORR): For the treatment in question, it was granted CMA based on an ORR of 32%. However, the confidence interval was notably wide, indicating substantial uncertainty about the effectiveness. - Subsequent Trials: The drug underwent further scrutiny in randomized controlled trials (RCTs) where it was compared to more standard treatments. The expectations were that it would demonstrate superior efficacy in terms of time to disease progression (TFS) and overall survival (OS).

Outcomes: - Trial Results: The results from the RCTs did not show the anticipated benefits. In fact, the hazard ratios for TFS and OS were slightly above 1, indicating no benefit and potentially even harm compared to the control. - Regulatory Actions: Given the disappointing results from the subsequent trials, the CMA was not renewed. This decision underscores the importance of robust post-approval studies to confirm initial findings.

Example in Primary Ciliary Dyskinesia

Background: Primary Ciliary Dyskinesia (PCD) is a rare genetic disorder that affects the cilia lining the respiratory tract, leading to recurrent respiratory infections, among other complications. Given its rarity, developing treatments for PCD presents unique challenges.

Challenges Faced: - Biomarker-Based Approval: The CMA for a PCD treatment was based on improvements in biomarkers rather than direct clinical outcomes, which can sometimes obscure the actual benefits to patients. - Treatment Response: The approval was based on a marked difference in biomarker response between the treatment and placebo groups—47% vs. 10%, which suggested a significant treatment effect.

Outcomes: - Subsequent Validation: The reliance on biomarker responses necessitates further validation to confirm that these improvements translate into real-world benefits such as reduced symptoms or improved quality of life. - Regulatory Scrutiny: The treatment continues to be evaluated, and the ongoing collection of data is critical to ensure that it meets the broader clinical needs of PCD patients.

whether fast access to medications is always beneficial

The question of whether fast access to medications is always beneficial is complex and merits consideration from multiple perspectives, especially regarding its impact on patients and society.

For Patients

Pros: 1. Immediate Relief: For patients suffering from severe, life-threatening, or debilitating conditions, fast access can mean quicker relief from symptoms and potentially life-saving treatment. 2. Hope and Psychological Benefit: Having rapid access to new treatments can provide hope and a psychological boost to patients who have exhausted other options, positively affecting their overall well-being.

Cons: 1. Safety Concerns: Fast-tracking can sometimes mean that not all potential side effects and long-term safety profiles are thoroughly vetted, which could lead to unforeseen adverse effects. 2. Efficacy Uncertainty: Quick approval may rely on limited or incomplete data, meaning the effectiveness of the treatment might not be fully established, which could lead to ineffective treatment plans.

For Society

Pros: 1. Healthcare Advancements: Fast access can accelerate the adoption of innovative treatments, pushing the healthcare industry forward and fostering a competitive environment that stimulates further research and development. 2. Economic Benefits: Successful new treatments can reduce long-term healthcare costs by curing diseases or reducing the need for ongoing treatment, besides contributing to the economic growth through pharmaceutical sales.

Cons: 1. Resource Allocation: Quick approvals might lead to significant investments in drugs with unproven long-term benefits, possibly diverting resources from other areas of healthcare that may offer more cost-effective benefits. 2. Regulatory Burden: Fast access can place a heavy burden on regulatory systems, requiring continuous monitoring and potentially leading to public health risks if not managed properly.

Conclusion

• The discussion emphasizes the critical role of post-approval studies in validating the efficacy and safety of drugs approved under CMAs. While these authorizations facilitate earlier patient access to important medications, they come with the responsibility to rigorously prove those drugs’ benefits in the post-market phase, ensuring public health safety and maintaining regulatory standards.
• These discussions underscore the complexity and potential pitfalls of relying on specific types of data for drug approval under CMAs. The balance between accelerating access to potentially life-saving treatments and ensuring that these treatments provide real, measurable benefits to patients is a critical concern. Regulatory bodies and the pharmaceutical industry must continue to refine their approaches to using and interpreting both clinical trial data and real-world evidence, ensuring robust and meaningful assessments of new therapies.
• Fast access to drugs is not inherently good or bad but comes with both significant benefits and potential risks. The key is finding a balance that maximizes benefits while minimizing harms. This involves robust post-marketing surveillance, transparent communication about the risks and benefits of new treatments, and ongoing research to fill in knowledge gaps about the effects of these rapidly approved medications. In essence, while fast access can be crucial in certain scenarios, it requires careful management to ensure that it truly serves the best interests of both individual patients and society at large.

These examples highlight several key points in the CMA process:

1. Importance of Rigorous Post-Approval Studies: Both examples illustrate why CMAs are often contingent on subsequent studies to validate initial findings. These studies are crucial to ensure that the benefits initially observed are real and meaningful.

2. Challenges with Surrogate Endpoints: Relying on surrogate markers or ORR can lead to approvals based on data that may not fully capture the drug’s impact on disease progression or patient quality of life.

3. Need for Holistic Assessment: These cases underscore the need for a comprehensive assessment that considers long-term outcomes and real-world effectiveness to truly gauge the value of new treatments.

2.3 S1-Talk 3: CLL11 – a trial tailored to answer questions from many stakeholders efficiently.

Speaker: Kaspar Rufibach (CH)

Overview of the CLL11 Trial

• Title: An open-label, multi-center, three-arm randomized, phase III study to compare the efficacy and safety of RO5072759 + chlorambucil (GClb), rituximab + chlorambucil (RClb), or chlorambucil (Clb) alone in previously untreated CLL patients with comorbidities.
• Design: Prospective, international, multicenter, open label, 3-arm randomized phase III study.
• Primary Endpoint: Progression-free survival (PFS).
• Secondary Endpoints: Included response rate (ORR/CR/PR), duration of response, disease-free survival in CR-patients, overall survival, minimal residual disease (MRD), safety profile, pharmacokinetics, and quality of life (measured using the EORTC questionnaire).

NCT01010061

Treatment Arms and Medications

• Arm A (GClb): Combined RO5072759 (GA101), a type of monoclonal antibody, with chlorambucil, administered intravenously and orally, respectively.

• Arm B (RClb): Combined rituximab, another monoclonal antibody, with chlorambucil.

• Arm C (Clb): Chlorambucil alone, serving as the control group.

• Involved 781 patients across multiple countries, including Germany, Australia, and the USA among others, highlighting its international scale and the broad applicability of its findings.

• Enrollment spanned from April 2010 to July 2012, with the study concluding in August 2017. The results were published in February 2018.

• The CLL11 trial’s results have had significant implications for the treatment of CLL, especially in patients with comorbid conditions. It supported the use of GA101 as a more effective treatment option, influencing treatment guidelines and practices worldwide.

• The trial was pivotal in obtaining approval for new treatment protocols that offer better management of CLL, leading to it being marked as a breakthrough therapy and changing the standard of care for CLL patients.

• The trial also contributed to the methodological discussion in clinical trials, particularly in the use of specialized statistical methods like close testing to handle multiplicity in data analysis, which ensures more robust and reliable conclusions.

Key Insights from the Presentation

• Underutilization of Available Tools: The speaker believes that statisticians often don’t fully exploit their analytical toolbox, defaulting to standard two-arm randomized controlled trials (RCTs) without considering more innovative or suitable methodologies that could address specific research questions more effectively.

• Stakeholder Engagement: Emphasizes the importance of convincing all stakeholders, not just regulators, about the value of a drug. This broader engagement is crucial for reducing time to market approval and reimbursement, which ultimately affects patient access to new therapies.

• Operational Considerations: Discusses the operational challenges and biases that were encountered during the trial, highlighting the need for robust discussions with regulators and developers to ensure trial integrity and acceptance of the results.

• Impact of the Trial: The trial led to the approval and reimbursement of a new treatment for chronic lymphocytic leukemia (CLL), marking it as a significant advancement in therapy. The trial’s design and outcomes were documented in clinical and statistical publications, although the speaker notes that these works are undercited and suggests they deserve more recognition.

• Multiplicity and Close Testing: The close testing approach used in the trial is praised for its efficiency in dealing with multiplicity, a common challenge in clinical trials where multiple comparisons increase the risk of type I errors. Despite its success, the method hasn’t been widely adopted, which the speaker attributes to a lack of awareness or understanding within the broader research community.

• Encouragement for Broader Adoption: The speaker advocates for a greater adoption of innovative statistical methods like close testing in clinical trials to improve efficiency and outcomes.

• Educational Push: Suggests that more educational efforts are needed to increase the visibility and understanding of such methods among statisticians and drug developers.

Make Who Happy

The discussion you provided revolves around the complex dynamics of drug development, especially from the perspective of ensuring comprehensive satisfaction across all stakeholders: regulators, patients, physicians, and payers (HTA bodies). It touches on the nuanced challenges of achieving regulatory approval while also meeting market and clinical demands for clear, comparative effectiveness. Here’s a breakdown of the essential points:

Addressing Multiple Stakeholders

• Regulatory Satisfaction: While regulatory approval is crucial, focusing solely on this aspect may not be sufficient. If the drug only marginally improves over existing treatments or does not address specific clinical questions, it may struggle in the market.
• Patient and Physician Acceptance: Patients and physicians look for clear benefits over existing therapies. In markets where a previous generation treatment (like the referenced Matera) is widely used off-label, new treatments need to offer distinct advantages to motivate a switch.
• Health Technology Assessment (HTA) Bodies: These organizations assess the value of new medical interventions in the context of their cost-effectiveness and broader impact on the health system. They require evidence that new treatments offer a clear benefit over existing options to justify the often higher costs associated with new drugs.

The Role of Trial Design in Drug Development

• Beyond Regulatory Approval: The speaker emphasizes that the real challenge in drug development is not just obtaining regulatory approval but ensuring the new treatment fulfills broader clinical and market needs.
• Efficient Trial Design: Discussing the potential for a three-arm trial involving the new therapy, the established standard, and possibly a previous generation treatment. This design aims to provide comparative efficacy data that can satisfy regulatory bodies, clinicians, and payers.

Close Testing Strategy

• Statistical Rigor: The speaker advocates for a “close testing” strategy, which is an advanced statistical method to manage multiple comparisons and control the overall type I error rate in a trial. This method allows testing of a global null hypothesis (that all treatments are equivalent), and if this is rejected, more specific pairwise comparisons can be conducted without inflating the type I error.
• Advantages of Close Testing: This approach is described as efficient because it systematically addresses multiple hypotheses, offering a robust way to demonstrate a drug’s effectiveness against various comparators. It’s framed as providing a “free lunch” in statistical terms — if the global hypothesis is rejected, the pairwise comparisons are effectively “unlocked” for testing at no additional penalty to the error rate.

Four Potential Strategies for Clinical Trials

1. Three Separate Trials:
   • Approach: Conduct three independent trials, each comparing two treatments with a significance level (α) set at 0.05.
   • Pros: Simplicity and clarity in evaluating each treatment comparison independently.
   • Cons: Inefficient in terms of patient resources as each patient is only used for one comparison, potentially requiring a larger total number of participants across all trials.
2. One 3-arm Trial with Bonferroni Correction:
   • Approach: Conduct a single trial with three arms and apply Bonferroni correction to control for multiple comparisons, reducing the α for each comparison to 0.0167.
   • Pros: More efficient use of patients as each participates in two comparisons, potentially reducing the total sample size compared to separate trials.
   • Cons: The stringent correction increases the difficulty of achieving statistical significance, possibly reducing the power of the trial.
3. One 3-arm Trial with Closed Testing, Wait until Last Comparison Mature:
   • Approach: Implement closed testing where the global null hypothesis is tested first; subsequent pairwise comparisons are tested only if the global null is rejected. All comparisons are analyzed once the data for the last comparison is mature.
   • Pros: Ensures that all necessary data are collected before making any comparative conclusions, potentially leading to more robust results.
   • Cons: Delays in final analysis as waiting for the last comparison to mature can significantly extend the study duration.
4. One 3-arm Trial with Closed Testing, Each Comparison Analyzed Once Mature:
   • Approach: Similar to the previous, but each pairwise comparison is analyzed as soon as it matures rather than waiting for all data to mature.
   • Pros: Allows earlier reporting of results for comparisons that mature faster, which could accelerate the process for successful treatments.
   • Cons: Risk of operational bias and complications from staggered analysis timings, which might affect the interpretation of results due to potential changes in trial or external conditions over time.

Choosing the Right Strategy

The choice among these strategies depends on various factors: - Efficiency: How quickly and efficiently can the trial achieve its objectives without compromising the integrity and reliability of the results? - Resource Allocation: How are patient resources best utilized to yield meaningful data? - Statistical Power: Which method provides enough power to detect real differences without excessive risk of Type I errors? - Regulatory and Stakeholder Expectations: What are the expectations from regulatory bodies and other stakeholders regarding the evidence needed to support claims of efficacy and safety?

Strategies and Their Time Implications

1. Three Separate Trials:

• Patients/HTA: Considered “hopeless” due to inefficiencies, as it takes a significantly long time to make all necessary comparisons to satisfy HTA bodies and patient groups.
• Regulatory Approval: Takes longer (34.4 months) than some more integrated approaches, due to the need to manage and synchronize separate sets of data and regulatory submissions.

2. One 3-arm Trial with Bonferroni Correction:

• Patients/HTA: Very lengthy (90.1 months), due to the stringent alpha adjustment required by the Bonferroni method, which can delay the achievement of significant results and thus slow down the process of satisfying HTA requirements.
• Regulatory Approval: Faster (21.4 months) than separate trials, as it allows for simultaneous comparison in a single cohort, albeit with stringent significance thresholds.

3. One 3-arm Trial with Closed Testing, Wait until Last Comparison Mature:

• Patients/HTA: Moderately quick (47.4 months), as this strategy waits for the last data set to mature before making any statistical comparisons, ensuring a comprehensive analysis but delaying final results.
• Regulatory Approval: Much slower (47.4 months) because it depends on the maturation of the last comparison, which may not be the most efficient for speedy approvals.

4. One 3-arm Trial with Closed Testing, Each Comparison Analyzed Once Mature:

• Patients/HTA: Faster (51.0 months) than waiting for the last comparison to mature, as it allows earlier testing of comparisons as soon as each respective dataset is ready.
• Regulatory Approval: The fastest method (18.6 months), allowing for dynamic and timely regulatory submissions as soon as initial significant results are available.

Conclusion

Efficiency: Closed testing where each comparison is analyzed once mature offers a balanced approach, providing relatively fast results for both regulatory approvals and satisfying HTA/patient concerns. It minimizes the delay usually associated with waiting for the last data point to mature (as seen in the last mature approach) and avoids the inefficiencies of running separate trials.

Statistical Power and Operational Bias: The approach with the least delay and operational bias while maintaining robust statistical power seems to be closed testing with each comparison analyzed once it matures. This strategy allows for a quicker response to emerging data without the stringent constraints of Bonferroni corrections.

Overall Consideration: A strategy that integrates closed testing with timely analysis of each mature comparison appears to be most advantageous. It aligns with regulatory timelines efficiently and addresses patient and HTA expectations more rapidly than other methods, thus facilitating a smoother and faster transition from trial completion to market availability.

2.4 S1-Talk 4: Fast and furious to market across Pharma, is it good for HTA?

Speakers: Karin Cerri and Lilla di Scala (J&J, CH)

Navigating Speed in Drug Development

Integration of Key Phases: - Approval: The first hurdle involves obtaining regulatory approval, which requires demonstrating safety and efficacy through rigorous clinical trials. Speed in this context means efficiently managing trial design and execution to meet regulatory standards without unnecessary delays.

• Access: Once approved, the focus shifts to market access, where the drug must be accepted into health systems and formularies. This involves negotiations with payers and proving the drug’s cost-effectiveness and benefit over existing therapies.

• Adoption: The ultimate goal is widespread adoption by healthcare providers and patients, which depends not only on the drug’s clinical benefits and cost but also on educational efforts and acceptance by the medical community.

Impact of Fast & Furious Study Designs: - The discussion also points out the implications of choosing faster, potentially less resource-intensive study designs, such as single-arm trials or non-randomized studies. These designs might speed up the process but can raise questions about the robustness of the data, potentially impacting market access and adoption due to less comparative and robust evidence.

Strategic Considerations: - Balancing Speed with Robustness: Achieving a faster timeline to market is crucial, especially in areas like oncology. However, the need for robust evidence that satisfies all stakeholders—regulators, payers, and the medical community—remains paramount.

• Tailored Approaches for Different Outcomes: Depending on the therapeutic area and the specific medical need, the approach to balancing speed and thoroughness may vary. For instance, in life-threatening conditions where no effective treatments exist, regulators and payers might accept faster-tracked approvals based on preliminary data.

• Continuous Data Collection: Post-approval studies and real-world evidence gathering are essential strategies to support fast-tracked approvals, ensuring that the therapies continue to demonstrate value in broader patient populations over time.

Concluding Thoughts: Fast-tracking drug development requires a strategic approach that considers how quickly a drug can move through approval, access, and adoption, ensuring that each step is optimized not just for speed but for sustainability and long-term impact. The ultimate success hinges on a well-coordinated effort among all stakeholders to ensure that accelerated pathways do not compromise the quality and thoroughness of the evidence needed to support new therapies. Thus, while speed is crucial, its direction and endpoint must be clearly defined to truly benefit patients and the healthcare system overall.

Detailed Consideration

1. Clinical Trial Designs and Their Evolution
   • Single-Arm Trials: These trials do not include a control group, which can make it challenging to evaluate the true efficacy of a treatment due to potential biases. They are often used when it’s unethical or impractical to withhold a promising treatment from patients (e.g., in life-threatening diseases where no standard treatment exists).
   • Placebo and Non-Randomized Studies: These can provide insights but are generally considered less robust than randomized controlled trials (RCTs) due to the potential for bias in assigning treatments.
   • Use of External Controls: This involves using data from outside the trial to form a control group, such as historical data or other databases. While this can expedite trials, it introduces complexities in ensuring the external data matches the trial population well enough to provide a valid comparison.
   • Adaptive and Phase 1 Extended Designs: Adaptive designs allow for modifications to trial parameters based on interim data, potentially speeding up development and making trials more flexible. Extended Phase 1 trials blend early safety assessments with efficacy studies typically seen in later phases.
2. Decision-Making in Trial Design
   • The design decisions influence the credibility and acceptability of trial outcomes for regulatory approval. This includes determining sample sizes, inclusion/exclusion criteria, endpoints, and whether interim analyses are conducted.
3. Trade-offs and Strategic Objectives
   • Trade-offs: Balancing scientific rigor against practical considerations such as patient recruitment and ethical concerns.
   • Strategic Objectives: Ensuring that the trial design addresses the regulatory requirements for demonstrating safety and efficacy, while also considering market access and reimbursement prospects post-approval.
4. Implications for Market Access and Treatment Adoption
   • The clinical trial outcomes must provide clear data on dosing, treatment duration, and side effects, which are crucial for securing payer coverage and influencing physician prescribing practices.
5. Role of HTAs in Evaluating Evidence
   • HTAs play a crucial role in determining whether a new treatment offers good value for money and should be reimbursed by health systems. They assess clinical benefit, cost-effectiveness, and sometimes wider social or ethical implications of new treatments.
   • There is a strong preference within HTAs for data from RCTs because these are viewed as the most reliable evidence due to their ability to minimize bias through randomization and blinding.
6. Methodological Considerations and Transparency in HTA Reviews
   • When trials use innovative or non-traditional designs (like external controls), transparency in how these designs were justified and their potential biases addressed is critical.
   • Challenges with External Control Arms: There must be rigorous justification for choosing this approach, and the comparability of external data with the trial population must be convincingly demonstrated.
7. Variability Across HTA Bodies
   • Different HTA organizations may have varying criteria and thresholds for what constitutes acceptable evidence, which can lead to different outcomes in terms of drug approval and reimbursement in different jurisdictions. ### Conclusion {.unnumbered}
8. Future Directions and Consistency Needs
   • There is a need for more standardized guidelines to reduce variability in how innovative trial designs are assessed by HTAs. This could include more consistent use of certain statistical methods or criteria for accepting external controls as valid comparators.
9. Conditional Approval and HTA Assessments
   • Conditional Approval: This refers to a regulatory decision to allow a drug to be marketed based on less comprehensive data than typically required. The approval is granted on the condition that further data from ongoing or additional studies will substantiate the initial findings and confirm the drug’s benefit-risk profile.
   • Impact on HTA: When a drug receives conditional approval, HTA bodies then have to evaluate the drug based on this preliminary data. This can introduce uncertainty into their decision-making process because the comprehensive data usually used to assess value and effectiveness is incomplete.
10. Longitudinal Analysis and Additional Benefit

• Long-Term Data: You mentioned that only 38% of drugs were found to have additional benefits over time, with this percentage even declining in some fields like oncology. This suggests that initial promising results might not always translate into long-term benefits, affecting how HTAs perceive these drugs.
• Uncertainty: The main takeaway here is that conditional approvals generate significant uncertainty for HTA bodies as they rely on complete and robust data to make informed decisions regarding drug reimbursement and adoption.

11. Comparative International Perspectives

• Studies from different countries (England, Scotland, France, Canada) show varied perceptions and outcomes concerning conditionally approved drugs. The variability in approval and rejection rates post-HTA suggests different standards or thresholds for efficacy and value across these regions.
• For instance, a small number of evaluations by NICE (the National Institute for Health and Care Excellence in the UK) might significantly influence the percentage interpretation of outcomes due to the low sample size.

12. HTA Decisions and Criteria for Evaluation

• Evidence Requirements: HTAs need solid evidence to assess a drug’s efficacy and safety. Conditional approvals, which are based on incomplete data, complicate these evaluations.
• Design and Methodological Concerns: Issues like inappropriate study design or the choice of comparators can lead to low ratings of clinical benefit by HTA bodies, as seen in some assessments where the social value or unmet medical needs of a condition did not outweigh the uncertainties in the presented data.

13. Time Delays and Re-Evaluation

• The gap between regulatory approval and HTA appraisal can be lengthy, especially when additional data are needed to satisfy the conditional aspects of the approval.
• This delay can mean significant time lost in terms of patient access to potentially beneficial treatments and can affect the overall lifecycle management of a drug.

14. Statistical Considerations and Implications for Policy

• The overlapping confidence intervals in data comparing expedited versus standard approval times suggest that, statistically, expedited approval might not significantly shorten the HTA evaluation timeline. This finding could influence future policy decisions on whether the expedited pathway offers a substantial benefit over standard procedures.

15. Expedited Pathways and HTA Challenges

• Regulatory vs. HTA Timelines: Even with expedited regulatory approvals designed to speed up the process due to urgent medical needs, HTA submissions and reviews often don’t significantly differ in timeline compared to standard procedures. This suggests that while regulatory bodies might fast-track approval, HTAs still require comprehensive data to make informed decisions about cost-effectiveness and reimbursement.
• Dosing Uncertainty: For patients, even if a drug is approved quickly, the uncertainty about the optimal dosing can be a significant issue, impacting treatment effectiveness and patient safety. This highlights the need for ongoing data collection and refinement even after a drug hits the market.

16. Specific Expedited Initiatives

• Orphan Drugs in Germany: Special provisions like those for orphan drugs in Germany, where drugs for rare diseases might receive more flexible or favorable HTA ratings due to lower alternatives and high medical need.
• NICE’s Proportionate Approach: Recently introduced frameworks aim to streamline the review process based on the drug’s expected impact and the quality of the evidence.
• Managed Access and Time-Limited Approvals: Some regions implement managed access agreements or time-limited approvals where drugs are allowed on the market conditionally while additional data is gathered over a set period (e.g., two years).

17. Building the Evidence Package Over Time

• Beyond RCTs: While randomized controlled trials (RCTs) are the gold standard, in the context of expedited pathways, additional types of data (real-world evidence, ongoing post-market studies) are increasingly crucial to address HTA’s concerns about long-term efficacy and safety.
• Strategic Data Collection: From early development through post-market, it’s vital to strategize what data are collected, how they are analyzed, and how they inform ongoing regulatory and HTA submissions.

18. Interconnectedness of Regulatory and HTA Processes

• Collaboration Between Regulators and HTAs: Encouraging dialogue and cooperation between regulatory bodies and HTAs can help align expectations and standards, facilitating smoother transitions from approval to reimbursement and access.
• Clear Goals: Understanding and clearly defining the end goals for expedited pathways—whether it’s faster patient access, addressing unmet medical needs, or optimizing health system resources—is essential for navigating these processes effectively.

Conclusion

• Strategic Partnerships: Working closely with HTAs from the outset can aid in aligning the drug development process with the evidential requirements of HTAs, potentially smoothing the pathway to market access.
• Patient Access Outcomes: Ultimately, the success of these pathways should be measured by their impact on patient access to effective treatments.
• Factoring in Time Loss Due to Re-submissions or Delays raises a critical question about whether the time lost due to necessary re-submissions or delays in HTA processes is formally accounted for in evaluations. This consideration is important for understanding the full impact of the HTA process on drug availability and patient access, particularly when initial submissions do not meet HTA standards.
• Opportunities Presented by Joint Clinical Assessments (JCA) JCAs offer a unique opportunity to streamline regulatory and HTA processes by providing a coordinated assessment. This can potentially reduce duplication of efforts and ensure that the regulatory packages are comprehensive and tailored to meet the fourth hurdle, which typically involves proving cost-effectiveness to HTA bodies.

3 Short topics: Present problem on 2-3 slides and receive input from a panel of regulators

3.1 Topic 1: Estimand Strategies for Handling Deaths in Early-Stage Neurological Disorder Studies

Judith Anzures-Cabrera, Annabelle Monnet, and Alex Strasak (Roche)

Study Details

• Recruitment: About 500 patients in the early stages of a neurological disorder are being recruited for the study.
• Expected Mortality Rate: The study anticipates a 1% mortality rate among participants.

An estimand is a precise description of what is being estimated from the data, which in this case assumes that death is the only intercurrent event (an event that occurs after treatment initiation and influences the interpretation of the primary outcome).

• Treatment A vs. Treatment B: The study will compare two different treatments to evaluate their efficacy.

Endpoints are the primary outcomes used to judge the effectiveness of a treatment. - Primary Endpoint: Time to a progression event. This means the main measure of treatment effectiveness is how long it takes before the disease worsens. - Key Secondary Endpoint: A continuous measure that is on the same scale as the time to event, providing additional information on treatment effects.

Strategies for Handling Death as an Intercurrent Event

This refers to how the study addresses deaths that occur during the trial, as these can impact the assessment of treatment effectiveness. - Time to Event (Composite): Death is considered a progression event. This approach allows the study to categorize any deaths as an endpoint, simplifying the analysis but possibly conflating death with other types of progression. - Continuous (Hypothetical): This method uses a statistical model to estimate what would have happened if the patient had not died. It tries to predict the outcomes as if the patients continued in the study, offering a way to account for deaths without counting them as a direct outcome measure.

Question

Question 1: Importance of Differentiating Unrelated Disease Progressions

Context: In your study, you’re using a composite strategy where death is counted as a progression event. The FDA has expressed concerns about this approach, possibly because it might conflate death with other progression metrics that are unrelated to the primary disease of interest.

Key Points to Address: - Differentiating Disease Progression: How crucial is it to distinguish between death and other progression events? This differentiation might impact how treatments are perceived, particularly in terms of their efficacy and safety. - Regulatory Perspective: Understanding the FDA’s standpoint on this differentiation could help refine your approach, ensuring it aligns more closely with regulatory expectations and provides clear, actionable data.

Question 2: Handling the Continuous Endpoint with Low Mortality Impact

Context: For your continuous endpoint, you’ve opted for a hypothetical approach, which models what would have happened if the patient had not died. Given the low mortality rate (1%), you’re seeking advice on whether this approach adequately addresses the potential distortion of treatment effects due to death.

Key Points to Address: - Feasibility of the Hypothetical Model: Is this model considered robust and valid by regulatory standards, especially with such a low incidence of the intercurrent event (death)? - Impact on Treatment Effect Estimates: How might the low mortality rate influence the reliability and precision of the hypothetical estimand in measuring the continuous outcomes related to symptom progression?

Feedback Q1

1. Uncertainty in Differentiation:
   • The feedback acknowledges that there is inherent uncertainty in differentiating whether progression events (like death) are related directly to the disease or the drug being tested, or if they are unrelated. This uncertainty complicates the assessment of both the disease’s natural progression and the treatment’s efficacy and safety.
2. Conservative Approach Recommended:
   • Due to the uncertainty mentioned, the feedback suggests a conservative approach, seemingly favoring not differentiating death as a progression event in the primary analysis. This approach would reduce the risk of incorrectly attributing death to the drug or disease, thereby potentially inflating efficacy or safety concerns artificially.
3. Study Design Implications:
   • It’s noted that ensuring the differentiation of progression events within the study is essential and would likely increase the complexity of the trial design. This might involve more detailed tracking and categorization of events, more complex statistical analyses, and potentially larger sample sizes to detect nuanced effects.
4. Competing Risks and Overall Survival:
   • The comment touches on the concept of competing risks, where the cause of death might not be directly related to the disease or its treatment, especially given it’s an early-stage disease with low mortality. The feedback suggests considering how these deaths relate to overall survival rates and whether they should be analyzed separately to understand their impact on treatment outcomes.
5. Importance of Understanding Death Causes:
   • It’s implied that understanding the specific reasons for deaths is crucial, even if they are infrequent. This understanding can influence how outcomes are interpreted in terms of both efficacy and safety.
6. Implications for Analysis:
   • There’s an indication that even if the mortality rate is low, the reasons behind any deaths should be thoroughly investigated and understood. This could mean differentiating between deaths caused by the disease, treatment, or other unrelated factors.

Feedback Q2

1. Challenge of Choice of Estimand:
   • The feedback emphasizes that while the choice of estimands (the specific ways in which you measure and interpret the data) should be challenged, this should only happen if there are issues with the estimations they produce. A strong, initial discussion about these choices is advocated to ensure that the decision-making is robust and well-justified.
2. Premature Simplification:
   • The commenter notes a tendency to simplify analysis approaches prematurely, perhaps due to perceived ease of implementation, before fully grappling with the complexities of the data or the nuances of what needs to be measured. This simplification might neglect critical aspects of the study’s findings.
3. Complexity of Neurological Function Assessment:
   • The feedback reflects concern over how neurological functions are quantified and related to clinical outcomes. The point here is that neurological functions can be affected by the disease or treatment in multiple ways, which necessitates careful consideration of how these effects are captured and interpreted in the study.
4. Statistical and Clinical Considerations:
   • There’s a distinction made between statistical and clinical questions. Statistical considerations often focus on the magnitude and detectability of effects, whereas clinical questions might delve into the broader implications of those effects on patient health and disease progression.
5. Handling of Low Mortality:
   • The feedback questions the impact of having a low mortality rate (less than 1%) on the study’s outcomes. It suggests that at such a low rate, the way deaths are accounted for might not significantly affect the overall results, though this could vary based on the disease and treatment mechanisms.
6. Composite and Hypothetical Strategies:
   • A suggestion was made to use a composite strategy for one type of endpoint and a hypothetical converter for the continuous endpoint. The rationale behind differentiating the treatment of death in these strategies seems to be questioned, particularly in light of the simplicity it might offer from a statistical perspective.

3.2 Topic 2: Estimation methods for estimands using the treatment policy strategy

James Bell (Elderbrook Solutions GmbH), Thomas Drury (GlaxoSmithKline), Tobias Mütze (Novartis Pharma AG), Christian Bressen Pipper (Novo Nordisk A/S), Marian Mitroiu (Biogen International GmbH), Khadija Rerhou Rantell (MHRA), Marcel Wolbers (Roche), David Wright (AstraZeneca)

Study Details

Handling missing data in clinical trials is a critical issue, particularly when endpoints are not observed due to subjects dropping out or other intercurrent events (IE). Addressing this missing data appropriately is essential to ensure reliable and accurate estimation of treatment effects.

• Missing Endpoint Data: When subjects leave a study prematurely, the intended endpoints are not observed, leading to missing data. This situation compromises the integrity of the statistical analysis and the conclusions that can be drawn about a treatment’s effectiveness.
• Importance of Appropriate Handling: Properly managing missing data is vital to avoid bias and to accurately estimate treatment policy effects.

Common Approaches to Handling Missing Data

1. Standard Methods:
   • These often do not adequately address the missing data problem, potentially leading to biased estimates.
2. Advanced Imputation Techniques:
   • Reference-Based Imputation: Techniques like “jump to reference” or “copy reference increments” assume that missing data after an IE can be approximated by observed data from a reference group. These methods require clear justification of the clinical assumptions made.
   • Retrieved Dropout Models: These models assume that missing data post-IE are similar to the observed data post-IE within the same study arm. They typically exhibit less bias in scenarios where the assumptions match reality.

Key Findings from Work

• Differences in Assumptions: The fundamental distinction between the two approaches lies in their assumptions about the nature of the missing data relative to observed data.
  • Retrieved Dropout Models assume continuity within the treatment arm, requiring no assumptions about treatment effects beyond observed changes.
  • Reference-Based Imputation assumes continuity with a reference group, imposing assumptions about treatment effects on the missing data.
• Statistical Implications:
  • Retrieved Dropout Models:
    • Pros: Typically show minimal bias in scenarios where sufficient data about the IE is available.
    • Cons: Require substantial data on IEs; without this, these models can lead to inflated standard errors and potential power loss.
  • Reference-Based Imputation:
    • Pros: Helps control type I error rate and doesn’t inflate standard errors.
    • Cons: Assumes specific treatment effects on the missing data, which can introduce bias if these assumptions do not hold true.

• Missing Data in Clinical Trials: The slides address the challenge of handling missing data due to intercurrent events (IE) like study withdrawal, which are common in clinical trials. Missing data can bias results and make the interpretation of treatment effects unreliable.
• Importance of Adequate Handling: Properly addressing missing data is crucial for accurately estimating treatment policy estimands, which are strategies for deciding on treatment approaches based on trial data.

Imputation Models Described: 1. Retrieved Dropout Models: - Assumptions: These models assume that missing data after an intercurrent event are similar to the observed data after such events within the same study arm. - Pros: - Minimal bias in realistic scenarios where assumptions hold. - No assumptions are made about the treatment effect beyond what’s observed. - Cons: - These models can have inflated standard errors and result in power loss when there is insufficient data post-IE. - They can be difficult or impossible to fit effectively if the data are sparse.

2. Reference-Based Multiple Imputation:
   • Assumptions: This approach assumes that missing data post-IE should be similar to observed data from a reference group, requiring explicit clinical assumptions and justification.
   • Pros:
     • Helps control the Type I error rate and avoids inflating the standard error.
   • Cons:
     • Assumes a particular treatment effect for subjects with missing post-IE data, which can introduce bias if these assumptions deviate from the true scenario.

Questions

Question 1: Acceptability of Reference-Based Imputation

Question 1: “When estimating a (primary) estimand that adopts a treatment policy strategy, would regulators accept an analysis approach that imputes the missing values using a reference-based imputation method if the assumptions of the imputation approach can be clinically justified?”

Context: This question concerns the regulatory acceptance of using a reference-based imputation method to handle missing data, particularly when such an approach aligns with the treatment policy strategy.

Key Considerations: - Regulatory Perspective: Regulators, like the FDA, generally require robust justification for any imputation technique used. The acceptability hinges on the clinical justification of the assumptions underlying the reference-based imputation method. - Clinical Justification: For a reference-based approach to be accepted, the assumptions about the similarity of missing data to the reference group must align with known clinical behaviors of the disease and treatment. It’s essential to demonstrate that the reference group is an appropriate and representative comparator.

Question 2: Principles Guiding the Selection of Imputation Model

• “Question 2 If it is unclear which assumptions are appropriate for the missing data imputation, which principles should guide the selection of the imputation model, e.g., type I error rate control, conservative bias for the treatment effect estimate (i.e., underestimate the treatment effect), bias-variance trade-off, clinical plausibility?”
  • “What is the priority order of the listed criteria?”
  • “Are there other important criteria which are not listed?”

Context: When it’s unclear which assumptions are most appropriate for the missing data, selecting an imputation model can be challenging.

Key Principles: 1. Type I Error Rate Control: Maintaining the integrity of statistical testing by controlling the false positive rate. 2. Conservative Bias for Treatment Effect: Preferring an underestimate of the treatment effect to avoid overestimating the drug’s efficacy. 3. Bias-Variance Trade-Off: Balancing the reduction of bias against the increase in variance, aiming to optimize the reliability of the estimate. 4. Clinical Plausibility: Ensuring that the imputation method produces results that are believable and align with clinical knowledge.

Priority Order: - Clinical Plausibility: Often considered a top priority because if the imputed data are not clinically plausible, the integrity of the entire analysis can be questioned. - Type I Error Rate Control: Critical for regulatory approval, as controlling the false positive rate ensures that findings are statistically sound. - Conservative Bias for Treatment Effect: Important in risk management, especially in clinical decisions where safety is paramount. - Bias-Variance Trade-Off: Important but often secondary to the above considerations, as a methodologically sound approach can still yield useful results even with some trade-offs in bias and variance.

Additional Criteria: - Robustness to Assumptions: The selected imputation method should be robust to variations in the underlying assumptions about the missing data. - Transparency and Reproducibility: The method should be clear and reproducible, allowing other researchers to understand and replicate the findings.

Feedback

Feedback to Question 1 (Imputation Methods and Regulatory Acceptance)

• Practicality and Justification: The feedback emphasizes that from a practical standpoint, using a reference-based imputation method is often recommended but must be thoroughly justified. It is not merely a matter of adopting a commonly recommended method; it’s about proving that this method fits the specific clinical scenario of the trial.
• Precision in Specification: It’s suggested that methods like reference-based imputation should be precisely specified and not allow for ad-hoc decisions during the trial. This pre-specification helps in maintaining the integrity of the trial and ensures that the method aligns well with regulatory expectations.
• Dynamic and Adaptive Approach: The discussion hints at the necessity of being adaptive in the trial execution, implying that while a plan should be rigidly predefined, responsiveness to emerging data and trial dynamics is also crucial.

Feedback to Question 2 (Principles Guiding Imputation Model Selection)

• Priority and Quantification: There’s a call for the quantification of priorities when selecting imputation methods, suggesting that different scenarios might require different priorities. This might involve balancing statistical needs against clinical practicalities.
• Handling Different Events: The feedback mentions the importance of understanding the nature of different intercurrent events and their impacts on the treatment or outcomes. The choice of imputation method should consider these differences to ensure that the estimand is appropriately reflective of the intended clinical question.
• Conservative vs. Unbiased Estimates: There is a nuanced discussion about the balance between conservative estimates (which might underestimate treatment effects to avoid overly optimistic conclusions) and unbiased estimates, which aim to reflect the true effect as accurately as possible.
• Multiple Estimates for Robustness: The idea of using more than one estimand to assess different aspects of the treatment effect is highlighted. This approach allows for a more comprehensive evaluation of the treatment and guards against biased conclusions based on a single analysis pathway.

Additional Insights:

• Importance of Follow-up: The feedback underscores the significance of follow-up in the trial, even post-intercurrent events, to ensure that all data contribute to the final analysis, thereby enhancing the robustness of the trial’s conclusions.
• Sensitivity Analyses: Suggests that sensitivity analyses are crucial, especially when assumptions about the imputation model cannot be verified. This helps in assessing the robustness of the conclusions under different scenarios.
• Role of Conservative Bias: The new discussion highlights scenarios where having a conservative bias in estimating treatment effects might be favored. This is particularly relevant in cases where the risks of overestimating the treatment’s effectiveness could lead to clinical decisions that might not be in the best interest of patient safety or efficacy. Conservative bias ensures that only treatments with clear, robust benefits are considered effective, reducing the risk of false positives.
• Clinical Sensibility and Treatment Context: Understanding the clinical context—such as the nature of the treatment and its expected impact on disease progression—is crucial in choosing the right imputation method. For example, the expected trajectory of disease progression (whether under placebo or active treatment) can guide the selection of imputation methods that best reflect realistic outcomes in the missing data.
• Different Estimates for Different Purposes: The discussion acknowledges that while the estimand might remain consistent, the specific estimates derived from it can serve different purposes. This differentiation might necessitate using various imputation methods or analytical approaches tailored to the specific objective of each estimate, whether it’s regulatory submission, clinical decision-making, or publication.
• Specification and Justification: There’s a reiterated emphasis on the necessity of clear specification and justification in the analysis plan. Every choice, especially concerning how missing data is handled, must be clearly justified based on the clinical setting and expected treatment dynamics.

Conclusion and Practical Application - Balance Between Statistical Rigor and Clinical Reality: The added discussion underscores the importance of balancing statistical rigor with clinical applicability. It suggests a dynamic approach where statistical methods are not only selected based on theoretical properties but also on how well they reflect and accommodate the realities of clinical practice. - Adaptive Strategies in Statistical Analysis: Given the complexity and variability in clinical trials, an adaptive approach to statistical analysis—responsive to emerging data and evolving clinical insights—is crucial. This approach ensures that the analysis remains relevant and robust across different stages of the clinical trial and subsequent regulatory review.

3.3 Topic 3: Two trials rule versus pooled trials rule

Fredrik Öhrn (J&J)

Study Details

The slides you provided discuss two major approaches in clinical trial design: the Two Trials Rule and the Pooled Trials Rule. These rules are strategies for designing the trial structure and analyzing data to ensure that the treatment effects are not only statistically significant but also unlikely to be due to chance.

Two Trials Rule

Concept: - This rule requires two separate trials to be conducted. Each trial independently tests the treatment’s efficacy against a control or placebo. - The primary endpoint of each trial is analyzed using a one-sided significance test with a threshold of \(p = 0.025\), aiming to maintain an overall type I error rate of 5% across the two trials.

Application: - This is a traditional approach in many therapeutic areas, especially where more robust evidence is needed due to variability in treatment response or criticality of the treatment outcomes. - Common in areas like oncology, rare diseases, and large cardiovascular trials.

Pros and Cons: - Pros: Provides independent replication of results, enhancing confidence in the treatment’s efficacy. - Cons: More resource-intensive; can be redundant if trials are highly consistent.

Pooled Trials Rule

Concept: - A single larger trial is conducted instead of two smaller ones. The primary endpoint analysis is still assessed at the level of \(p = 0.025^2\) (i.e., 0.000625), to achieve the same overall type I error control as two separate trials. - This approach assumes the total sample size of the single trial equals the combined sample size of two individual trials (2N).

Application: - This method is suited for scenarios where pooling data into one larger trial can provide a clearer, more comprehensive understanding of the treatment effects and allow for early stopping and other interim analyses.

Pros and Cons: - Pros: - Potential for moderate power gains due to larger sample size in a single analysis. - Facilitates early stopping and other interim adaptations, which can save resources if the trial can be concluded early. - Streamlines operations by focusing on a single trial setup. - Cons: - Risks associated with basing decisions on a single trial, which could be more affected by an unusual outcome. - Concerns that it might motivate smaller, less robust study designs.

Question

The Proposal: 1. Two Separate Trials (each with sample size \(N\)): Traditionally, conducting two independent trials provides a robust means of validating the treatment’s efficacy by allowing independent replication of the results. This method is particularly valuable in proving consistent effects across different study populations or conditions and is often used to satisfy regulatory requirements for drug approval.

2. A Single Larger Trial (with sample size \(2N\)): This alternative suggests pooling resources into one larger trial, which could potentially increase the trial’s power and enable more complex statistical analyses, such as interim analyses or adaptive trial designs. The single larger trial approach might also be more efficient in terms of resource utilization and time.

The Question:

In which situations would it be appropriate to replace the two-trial approach (1) with a single, larger trial (2)?

• Statistical Power: A single trial with a larger sample size (\(2N\)) may have greater statistical power to detect a treatment effect, especially if the effect size is small. This setup can be crucial in studies where the expected difference between treatment and control is subtle but clinically significant.

• Resource Constraints: If resources are limited, conducting one larger trial might be more feasible than managing two separate trials. This can include financial constraints, limited patient populations (especially in rare diseases), or limited timeframes for study completion.

• Regulatory and Strategic Considerations: If the regulatory body is open to innovative trial designs and the primary concern is demonstrating a strong, statistically significant result, one large trial might suffice, especially if combined with robust interim analyses that could further validate the findings without the need for a second trial.

• Homogeneous Population: If the population is relatively homogeneous and external validity (generalizability across different populations) is less of a concern, a single trial might be more justified.

• Interim Analyses and Early Stopping: Large single trials often facilitate the use of interim analyses, which can allow for early stopping for efficacy, thus potentially bringing effective treatments to patients sooner.

Feedback

1. Value of Replication:
   • Replication Importance: The value of replication, specifically through independent trials, is emphasized as crucial for ensuring the robustness of findings. Replication helps confirm that results are not due to peculiarities of a specific study setting or sample.
   • Variability Across Trials: Having two trials allows for the assessment of variability and consistency across different settings or populations, enhancing the credibility of the findings.
2. Statistical and Operational Considerations:
   • Sample Size and Power: A larger single trial could theoretically offer similar statistical power to two smaller trials but with the added benefits of operational efficiencies and potentially faster completion times.
   • Internal Replication: The possibility of internal replication within a larger trial is mentioned, where different segments or regions within the trial could serve as semi-independent assessments of the treatment effect.
   • Interim Analysis and Adaptations: Larger trials often allow for more flexible interim analyses, including possibilities for early stopping if efficacy is clearly demonstrated or if safety concerns arise.
3. Biases and Risks:
   • Bias Mitigation: Multiple trials can help mitigate biases that might occur from a single trial’s specific methodological or operational issues.
   • Risk of False Positives/Negatives: The discussion acknowledges the risk of incorrect conclusions from a single trial due to its larger influence by any single set of conditions or anomalies.
4. Practical Implementation and Preferences:
   • Preference for Two Trials: Despite the potential benefits of a single large trial, there’s a strong preference for two trials due to the added assurance they provide through replication.
   • Regulatory Considerations: Regulatory preferences may also dictate the need for multiple trials, especially in high-stakes drug approvals where robust evidence is required.

Phase 2 to 3

1. Replication from Phase 2 to Phase 3:
   • The speaker highlights the importance of replication from phase 2 (exploratory) studies to phase 3 (confirmatory) trials. This is crucial because phase 2 trials are generally designed to explore therapeutic efficacy and optimal dosing regimens, whereas phase 3 trials are confirmatory in nature and designed to substantiate the efficacy and safety findings from phase 2.
   • The suggestion is that having a replication of findings from phase 2 in phase 3 adds robustness and confidence in the results, thereby supporting regulatory approval.
2. Value of Twin Trials in Confirmatory Phases:
   • The preference for “twin trials” (conducting two independent trials) in the confirmatory phase is emphasized because it provides additional validation of the treatment’s effectiveness and safety. This is particularly preferred if the initial trials yield marginal or ambiguous results or when the treatment effects between different doses aren’t well understood.
3. Biases Towards Favorable Outcomes:
   • There’s an acknowledgment that stakeholders, including developers and possibly regulators, might be biased towards approving treatments, especially in disease areas where there are limited or no existing treatments. This bias can sometimes overshadow objective assessment, leading to a preference for believing in the efficacy of the treatment even if the data are not conclusively supportive.
4. Reflection on Clinical Trial Guidelines:
   • Mention of a “reflection paper” suggests there are documented considerations or guidelines that outline key points to consider during trial design and evaluation. These might include statistical considerations, ethical issues, or practical aspects related to trial execution.

Summary of Feedback: The feedback converges on a few critical themes:

• Dependence on Context: The decision between one or two trials heavily depends on the specific drug, disease area, and regulatory environment. Certain conditions, such as rare diseases or areas with high variability in treatment response, might particularly benefit from multiple trials.
• Operational and Strategic Flexibility: While two trials provide robustness, a single larger trial might be more feasible and efficient in some contexts, especially when resources or suitable patient populations are limited.
• Statistical Integrity vs. Operational Efficiency: Balancing the statistical integrity provided by multiple trials against the operational efficiency and potential cost savings of a single trial is a central theme.

3.4 Topic 4: Testing procedure for multiple treatments and multiple outcomes

Marc Buyse and Samuel Salvaggio (One2Treat)

Study Details

The provided images outline the design and multiple testing procedures for a clinical trial focused on advanced cancer, involving three arms: two experimental (A and B) and one control (C).

• Trial Arms:

  • A (Experimental 1): This arm tests a specific experimental treatment.
  • B (Experimental 2): This arm tests another experimental treatment.
  • C (Control): This arm uses a standard treatment or placebo as a control.

• Randomization: Patients are randomly assigned in a 1:1:1 ratio to one of the three arms.

• Comparisons:

  • A vs. C: This is the primary comparison where Experimental 1 is compared against the control.
  • B vs. C: This is another key comparison where Experimental 2 is compared against the control.
  • A vs. B: This comparison is not powered, meaning it is not the primary focus of the study and may not have enough participants to conclusively detect differences between A and B.

• Outcomes:

  • Primary Outcome: Progression-Free Survival (PFS), which measures the length of time during and after treatment that a patient lives with the disease but it does not worsen.
  • Secondary Outcome: Overall Survival (OS), which measures the length of time from either the date of diagnosis or the start of treatment that patients diagnosed with the disease are still alive.

Multiple Testing Procedure

• Testing Strategy:
  • The primary endpoint PFS for Arm A vs. C is tested first with an alpha level of 0.05.
  • If PFS for Arm B vs. C shows a positive result, the OS for Arm A vs. C is tested at an alpha of 0.01.
  • Depending on the outcomes of these tests, subsequent tests are adjusted:
    • OS for Arms B vs. C: Tested if PFS (B vs. C) is not positive and OS (B vs. C) is positive with varying alpha levels.
    • The alpha level for testing OS varies based on the results of PFS analyses. If both are positive, more stringent criteria are applied.
• Issues:
  • OS is only tested for Arm A vs. C if PFS for Arm B vs. C is statistically significant.
  • This testing procedure ensures that the study maintains its overall type I error rate across multiple comparisons and endpoints, thereby controlling for the possibility of false positives due to multiple tests.

Question

The proposal you’ve shared suggests an alternative approach for a clinical trial design focusing on advanced cancer, shifting from using Progression-Free Survival (PFS) to prioritizing outcomes like Overall Survival (OS) and Time to Tumor Progression (TTP).

Alternative Approach: Generalized Pairwise Comparisons (GPC) 1. Shift in Primary Outcomes: - Time to Death (OS): This measures the duration from the start of the trial until death from any cause, providing a direct indication of a treatment’s effect on survival. - Time to Tumor Progression (TTP): This tracks the time until the tumor shows signs of progression, which can be a more immediate and clear measure of a treatment’s efficacy compared to PFS.

2. Analysis Strategy:
   • The proposal suggests analyzing these outcomes using Generalized Pairwise Comparisons for the two key comparisons in the trial: A vs. C and A vs. B. This method likely aims to provide a direct and specific comparison between treatments, enhancing the clarity and applicability of the results.
3. Statistical Adjustment for Multiplicity:
   • Methods like Holm’s or Hochberg’s procedures are recommended to adjust for the problem of multiple comparisons within the trial. These procedures are designed to control the family-wise error rate, ensuring that the probability of making one or more type I errors among all the hypotheses tested is kept within a desired level.

Questions Regarding the Alternative Approach:

1. Complexity and Acceptability: The first question raises concerns about the complexity of explaining this approach and whether it would be acceptable to regulatory agencies. Given that regulatory bodies often require clear, robust evidence to support approval, the use of prioritized outcomes that directly reflect patient survival and disease progression may indeed be compelling. However, the shift from a more traditional endpoint like PFS to TTP and OS could require additional justification to demonstrate that these measures effectively capture the treatment benefits.

2. Feasibility of Implementation: The second question implicitly asks about the feasibility and practical challenges of implementing such a trial design. This involves considerations such as:

   • Whether the data collection for OS and TTP can be reliably performed within the trial constraints.
   • How well these outcomes can be statistically analyzed and interpreted in the context of the trial’s goals.
   • The potential need to explain and justify the use of multiple testing adjustments like Holm’s or Hochberg’s methods, which might be less familiar to some stakeholders compared to more straightforward methods.

Feedback

1. Complexity of Explaining Trial Designs to Patients

• Patient Comprehension Issues: Discussing different endpoints like PFS and OS with patients is challenging due to their varying levels of intuitiveness and direct relevance to patient outcomes. PFS might not resonate as significantly with patients as OS, which is more directly related to survival and thus easier for patients to understand.

2. Endpoint Selection and Statistical Management

• Multiple Endpoints: Incorporating multiple endpoints adds complexity, raising questions about why certain endpoints are prioritized over others. This complexity necessitates advanced statistical methods, such as Holm’s procedure, to manage the risk of type I errors across multiple tests.
• Strategic Endpoint Ordering: The sequence of testing endpoints is designed to maximize the chances of demonstrating treatment efficacy early on. While this strategy might boost a trial’s efficiency in showcasing results, it complicates the interpretation and might obfuscate the overall understanding of the trial’s outcomes.

3. Challenges in Patient-Centric Communication

• Expertise Requirement: Effective communication and justification of trial designs exceed the typical understanding of standard patients. This gap indicates a need for strategies that enhance patient-centric approaches, ensuring patients fully grasp the trial’s intentions and implications.
• Statistical Concepts: Explaining complex statistical concepts related to multiplicity and error control to patients is particularly challenging. These concepts are crucial for the scientific integrity of the trial but are often difficult for patients to understand due to their technical nature.

4. Ethical and Strategic Considerations

• Balancing Goals: There is an ongoing philosophical debate about whether clinical trials should focus more on patient support and understanding or on advancing drug development objectives. This discussion emphasizes the need to find a balance that respects both ethical considerations and the practical demands of drug development.
• Trade-offs in Design Choices: Selecting endpoints involves significant trade-offs, highlighting the difficulty in balancing simplification for patient understanding with the need for rigorous scientific validation. This complexity calls for a thoughtful approach that aligns clinical trial designs with both scientific standards and patient welfare.

3.5 Topic 5: What are the Quality Standards for Exploratory Analyses?

Kostas Sechidis, Mark Baillie, and Bjorn Bornkamp (Novartis)

Study Details

The PPDAC framework, which stands for Problem, Plan, Data, Analysis, Conclusion, serves as a structured approach to ensure clarity, reliability, and replicability in data analysis and decision-making. This framework is rooted in statistical thinking and aims to guide researchers systematically through each phase of their investigation.

1. Problem (P): Identifying and pre-specifying the research questions or problems that need to be addressed. This involves defining the target estimand that the analysis will estimate.

2. Plan (P): Developing a comprehensive plan that aligns the questions with appropriate data collection and analysis strategies. This ensures that the data collected are suitable for addressing the research questions.

3. Data (D): Gathering and initially analyzing the data to understand the context, potential pitfalls, and the basic properties of the collected information.

4. Analysis (A): Implementing the analysis in a reproducible and accurate manner, ensuring that the methods and procedures used can be repeated and verified by other researchers.

5. Conclusion (C): Reporting the findings clearly and transparently, making it easy for other stakeholders to understand the results and the implications.

Application to Regulatory and Exploratory Analysis

• Versatility of PPDAC: This cycle is applicable to both exploratory and confirmatory analyses, making it a robust framework for various phases of research, from initial hypothesis generation to final results confirmation.

• Exploratory Assessments: Specifically, in the context of treatment effect heterogeneity, the PPDAC framework can help identify and explore variations in how different subgroups respond to a treatment within the exploratory phases of clinical research.

Purpose and Importance of Exploratory Analyses - Crucial Role in Drug Development: Exploratory analyses help address important questions that influence decisions on drug approval and labeling. These include identifying covariates for adjustment, assessing treatment effect heterogeneity, and conducting safety assessments. - Flexibility and Rigor: While these analyses offer the flexibility necessary for exploration and learning from data, they require increased self-discipline and rigor to ensure the outcomes are reliable and valid.

Challenges in Exploratory Analysis - Lack of Clear Guidelines: Unlike primary analyses, exploratory analyses often lack well-established guidelines, making the process less clear and potentially variable in quality. - Regulatory Interest and Comparisons: There is interest in creating structured guidelines similar to those in other fields such as pharmacometrics (Model Informed Drug Development, MIDD) and machine learning in medical devices (Good ML Practice for Medical Device Development). These frameworks provide structured approaches to integrating modeling and simulation into drug development and regulatory submissions.

Question

The main question raised in the discussion pertains to the establishment of quality standards for exploratory analyses in drug development from a regulatory perspective. Specifically, the query seeks to open a dialogue on:

What are the quality standards for exploratory analyses?

This question aims to address the lack of clear, established guidelines for conducting exploratory analyses in drug development, which are crucial for making informed decisions about drug approval and labeling. The question underscores the need for a structured framework that ensures the reliability and validity of the results derived from such analyses.

• Opening a Discussion on Quality Standards: The text invites a discussion on what the quality standards should be for exploratory analyses, emphasizing the need for a framework that could guide researchers effectively through the necessary steps of an exploratory study.
• Example of a Structured Framework: The text references the PPDAC cycle (Problem, Plan, Data, Analysis, Conclusion) as a potential model for structuring exploratory analyses. This cycle encourages systematic progression through:
  • Defining the problem or research question.
  • Planning the analysis including methods and data needs.
  • Collecting and preliminarily analyzing the data.
  • Conducting the analysis in a reproducible and accurate manner.
  • Concluding with clear and transparent reporting of the findings.

Feedback

1. Utilization of Existing Frameworks:
   • Existing frameworks that incorporate modeling and simulation are advocated for effectively addressing both research questions and regulatory requirements, highlighting their integral role in the exploratory analysis phase of drug development.
2. Promotion of Credibility Framework:
   • The credibility framework is emphasized as a foundational starting point, valued for its comprehensiveness in covering research and regulatory aspects, ensuring that exploratory analyses maintain rigor and meet regulatory expectations.
3. Focus on Safety and Efficacy:
   • There is a strong advocacy for focusing intensively on safety and efficacy through structured exploratory analyses. This approach is essential for early identification of potential issues, refining the drug development process to better meet regulatory standards and enhance patient outcomes.
4. Collaboration Between Sponsors and Regulators:
   • The importance of ongoing dialogue and collaboration between drug developers and regulatory bodies is underscored, highlighting that such partnerships are vital for aligning scientific findings with regulatory guidelines and safety considerations.
5. Role and Adaptability of Exploratory Endpoints:
   • Exploratory endpoints are not merely for fulfilling regulatory requirements but are crucial for refining understanding of treatments at various developmental stages. The need for these endpoints to be dynamically relevant and adaptable as new data emerge is stressed, particularly in early-phase trials.
6. Importance of Relevant Information:
   • The continuous integration and effective communication of relevant information are highlighted as vital for ensuring that all stakeholders, including regulatory bodies, fully understand and support updates and changes in endpoint strategies.
7. Discussion and Stakeholder Engagement:
   • Engaging a broad range of stakeholders in discussions about endpoint strategies and exploratory analyses is crucial. These discussions help ensure that changes in endpoints are data-driven, well-supported, and aligned with clinical and regulatory goals.
8. Contextual Relevance of Analyses:
   • The applicability of exploratory analyses extends beyond conventional scenarios, encompassing areas like infectious diseases and various disease progression stages. This underlines the necessity for endpoint strategies that are specifically tailored to the therapeutic context and patient needs.

3.6 Topic 6: Contribution of Sequence

Hong Sun (BMS)

Study Details

• Neoadjuvant → Surgery → Adjuvant: This is the typical sequence in many clinical trials involving surgery. Patients receive a neoadjuvant treatment (such as Drug X or standard of care (SoC)) before surgery. After surgery, patients continue with adjuvant treatment (either Drug X or SoC) during the post-surgical phase. The study also involves follow-up to monitor outcomes like recurrence and overall survival.

• Radiologic Restaging: After the neoadjuvant treatment and before surgery, radiologic imaging is performed to assess how well the tumor has responded to the initial treatment. This information helps inform the surgical plan.

Study Design Options to Address the Contribution of Sequence:

• Factorial Design or 3-arm Design: The FDA recommends using a factorial design or at least a 3-arm study design in peri-operative settings to clearly understand the effects of the adjuvant treatment and to address potential safety concerns like overdosing.

  • Factorial Design: In this design, all possible combinations of treatments are evaluated. For example, patients may receive Drug X both before and after surgery, Drug X only before surgery, Drug X only after surgery, or no Drug X (control or SoC group). This design helps isolate the specific contribution of each treatment period (neoadjuvant or adjuvant) to overall treatment efficacy.

  • 3-Arm Design: In a simpler 3-arm design, patients are randomized into three groups:

    1. Drug X during both neoadjuvant and adjuvant periods.
    2. Drug X during the neoadjuvant period and SoC after surgery.
    3. SoC during both neoadjuvant and adjuvant periods.

  This design allows researchers to compare the effects of different treatment sequences more straightforwardly.

FDA Recommendations:

• Safety Concerns and Overdosing: To avoid overdosing in peri-operative settings, the FDA, during the ODAC (Oncologic Drugs Advisory Committee) meeting on July 25th, 2024, recommended using a factorial or 3-arm design to help clearly differentiate the effects of the treatments administered at different phases of the peri-operative period and to monitor safety outcomes more effectively.

Question

1. FDA’s Recommendation for Factorial or 3-arm Design:
   • Question: What are the recommendations from other regulatory agencies?
     • Explanation: While the FDA recommends using a factorial design or at least a 3-arm study design in peri-operative settings to assess treatment effects and avoid potential safety concerns (like overdosing), this question seeks to know if other regulatory agencies (e.g., EMA, PMDA) provide similar or different guidelines for such study designs in peri-operative trials. Understanding global regulatory expectations is critical for ensuring that trials comply with various jurisdictions.
2. Choice of Treatment Regimen for Sequence Comparison:
   • Question: Assuming that a 3-arm design approach is pursued, which type of treatment regimen should be compared with peri-adjuvant/peri-operative regimen: neoadjuvant only or adjuvant only for assessing the contribution of sequence?
     • Explanation: This question is about selecting the appropriate comparison group when assessing how much the treatment sequence (peri-adjuvant or peri-operative) contributes to overall efficacy. Should trials compare the peri-adjuvant regimen to just a neoadjuvant-only regimen, an adjuvant-only regimen, or something else? The goal is to isolate the specific effects of treatment timing and understand which sequence provides the most significant clinical benefit.
3. Feasibility of Using a Phase-2 Study with 3 Arms:
   • Question: Can a phase-2 study with 3 arms be used for demonstrating the contribution of sequence?
     • Explanation: This question explores whether it is feasible to use a phase-2 study, which is typically smaller and earlier in the development process, with three arms to effectively demonstrate how the sequence of treatment (e.g., neoadjuvant vs. peri-adjuvant) contributes to overall outcomes. It raises the issue of whether phase-2 trials are adequate in terms of power and design to provide meaningful insights into the sequence’s role.
4. Demonstrating Treatment Effects in a 2-arm Design:
   • Question: In a peri-operative setting with only 2 arms (peri vs. SoC), could we demonstrate the treatment effect from the neoadjuvant period or adjuvant period using statistical methods?
     • Explanation: If a study only has two arms (peri-adjuvant vs. SoC), this question asks if statistical methods can be used to still demonstrate the independent contributions of the neoadjuvant or adjuvant periods to overall treatment effects. For example, statistical techniques like:
       • Landmark analysis: Evaluating outcomes from surgery or another specific time point.
       • Indirect comparison: Using propensity score analysis, external controls, or historical data to compare the treatment effects indirectly.
       • Responder or subgroup analysis: Analyzing specific groups of patients who respond better to certain treatment timings.
5. Guidelines for Studies Involving Treatment Sequences:
   • Question: Are there any draft/final guidelines that we can refer to for the new and existing studies involving different treatment sequences or treatment phases?
     • Explanation: The final question seeks information on whether any formal guidelines, either in draft or final form, are available that address studies with different treatment sequences or phases. This would help researchers and sponsors align their studies with regulatory expectations and best practices for trials that investigate treatment timing (e.g., neoadjuvant, adjuvant, peri-operative therapies).

Feedback

Re-randomization involves randomizing participants again at a later stage of the trial, typically after completing an initial phase of treatment (e.g., after surgery or after neoadjuvant treatment). This method allows researchers to assess the impact of the subsequent treatment phases (such as adjuvant therapy) more clearly by reducing confounding factors and isolating the effects of different treatments or sequences.

One of the primary challenges in clinical trials with peri-operative settings is disentangling the effects of treatments administered at different times (neoadjuvant vs. adjuvant). By re-randomizing participants after a significant treatment milestone, like surgery, the study can separately evaluate the effects of each phase. For example: - Initial randomization could occur before surgery, where participants are assigned to different neoadjuvant treatments. - Re-randomization after surgery would then assign participants to different adjuvant treatments, allowing the effects of the adjuvant phase to be assessed separately from the neoadjuvant phase.

Re-randomization helps reduce potential biases that might arise if the effects of neoadjuvant treatments influence the outcomes of subsequent phases. By re-randomizing, the researchers can better control for the impact of the initial treatment phase and isolate the contribution of later phases (e.g., adjuvant therapy). This separation can clarify the sequence’s role in the overall treatment effect, offering a more rigorous evaluation of each phase’s contribution.

One of the concerns mentioned in the discussion is the challenge of underpowered studies, especially when it comes to understanding sequence contributions. By re-randomizing participants and collecting data on different treatment regimens at various stages, the trial may gain more statistical power to detect differences between sequences. This approach could lead to more robust conclusions about the most effective timing and combination of treatments.

Challenges and Considerations While re-randomization offers potential benefits, it also introduces complexity to trial design: - Sample size and logistics: The study may require a larger sample size to maintain sufficient power across multiple randomizations. - Patient adherence: Participants might be less willing to adhere to treatment if they feel uncertain about being re-randomized after a significant treatment phase like surgery. - Safety concerns: If there are long-term safety concerns related to one of the treatment phases, re-randomization might make it harder to fully assess the long-term effects of earlier treatments.

1. What are the recommendations from other regulatory agencies?
   • The discussion highlights the difficulty of addressing the contribution of different treatment phases (e.g., neoadjuvant, adjuvant) in the trial designs. The complexity of establishing a clear contribution from each phase makes it hard to obtain a specific recommendation from other regulatory bodies. However, any approach that attempts to clarify this contribution is seen as helpful, and it may vary between agencies. This underscores the importance of a thoughtful design that takes into account specific regulatory expectations.
2. Which type of treatment regimen should be compared with peri-adjuvant/peri-operative regimen (neoadjuvant only or adjuvant only)?
   • The discussion suggests that pulling treatment arms in a trial might provide more insights into how different treatment phases contribute to overall success. A more natural choice could be a study that compares the combination of neoadjuvant and adjuvant regimens to isolated regimens. This would give a clearer understanding of which sequence or combination of treatments is most beneficial to patients. However, the challenge remains in defining the contribution of each phase.
3. Can a phase-2 study with 3 arms be used for demonstrating the contribution of sequence?
   • There is skepticism about the ability of a phase-2 study to provide meaningful conclusions regarding the contribution of treatment sequences. The endpoints typically used in phase-2 trials may not allow for a deep understanding of sequence contribution, particularly because many concerns arise from safety data. Additionally, in phase-2 trials, it’s difficult to disentangle the effects of different treatment phases, and long-term toxicity data is often missing. Therefore, it may not be the best design for assessing sequence contribution.
4. Could we demonstrate the treatment effect from neoadjuvant or adjuvant period using statistical methods (with only 2 arms in the peri-operative setting)?
   • While the question remains open, the discussion reflects that using statistical methods like landmark analysis, indirect comparisons, or responder analyses could be possible in some cases, but it’s a difficult and complex process. Additionally, leveraging information from a phase-3 trial to inform the analysis might be helpful. However, the uncertainty about how much can be solved in a phase-2 study leaves this approach as challenging and not always effective.
5. Are there any draft/final guidelines for studies involving different treatment sequences or treatment phases?
   • The discussion does not directly answer whether there are draft or final guidelines available, but it implies that this is a recurring challenge that comes up in scientific advice discussions. It suggests that there are no clear, universally accepted guidelines, and the design approach may depend on the specific case and needs of the study. The complexity and lack of a clear regulatory framework for different sequences in treatment phases make this an evolving area in clinical trial design.

4 Session 2: Estimands – Celebrating the 5th anniversary of ICH E9(R1) – what have we learned and where do we need to go?

Chairs: Andreas Brandt (BfArM, DE) and Vivian Lanius (Bayer, DE)

While the estimand framework has been implemented across trials, the speaker feels that its full potential has not been realized. Its true power lies in helping researchers and regulators focus on the core questions of what they want to learn about a drug, why they are conducting the trial, and how to best interpret the results for clinical practice. Using the framework to get to the heart of these questions is essential to maximizing its impact.

4.1 S2-Talk 1: Estimands: Implemented, but not fully embraced.

Speakers: Frank Bretz (Novartis, CH, virtual) and Rob Hemmings (Consilium, UK)

Key Concepts

• Historical Context and Acknowledgment: The presentation highlights how estimates have evolved over the last decade, with early discussions about how to incorporate real-world data into trial designs and how to handle intercurrent events. Over the years, there has been more acknowledgment of these complex issues and a better understanding of how to manage them in clinical trials.

• Data Analysis and Decision Making: The emphasis is on the fact that clinical trial analysis involves making choices about how to deal with intercurrent events and how to align those choices with the underlying clinical questions. The example given refers to how early discussions might have ignored data or not properly aligned the clinical and analytical decisions, but improvements have been made.

• Regulatory Focus: Regulatory bodies, such as the FDA, have been involved in these conversations and highlight the importance of aligning strategies with clinical outcomes and patient retention. The estimation process and the strategies used must be transparent and justifiable.

• Positive Reflection: The discussion reflects a positive development in how researchers are now tackling complex problems, such as intercurrent events, more openly and thoughtfully. There are still challenges, but the conversations are progressing, particularly in difficult-to-analyze datasets (like those involving early progression or deaths in neuroscience trials).

1. ICEs (Intercurrent Events):
   • Intercurrent events are events that occur after treatment initiation that could affect the interpretation of the clinical trial’s outcomes. These events could include things like patients discontinuing treatment, switching therapies, or experiencing unrelated health events.
   • The slide indicates multiple intercurrent events (e.g., Intercurrent Event #1, Intercurrent Event #2), which require specific strategies to handle them during data analysis.
2. Strategies:
   • The strategies for handling intercurrent events are the decisions made to account for these events when analyzing the trial data. Common strategies include:
     • Treatment policy strategy: Continuing to analyze the outcome regardless of the intercurrent event.
     • Hypothetical strategy: Analyzing what the outcome would have been if the intercurrent event hadn’t occurred.
   • Different intercurrent events may have different strategies depending on their impact on the clinical outcome.
3. Attributes:
   • Population: Refers to the group of patients being studied.
   • Treatment Conditions: The treatments being administered in the study (e.g., drug, placebo).
   • Variable: The outcome variables being measured (e.g.,

Treatment-Policy Strategy

Objective:

revolve around analyzing whether the commonly used “treatment-policy strategy” in clinical trials truly reflects the underlying clinical question of interest and promoting a broader, interdisciplinary conversation around estimand implementation.

The goal is to: - Highlight key considerations: The slide aims to use examples (though non-exhaustive) to highlight critical points researchers and stakeholders should consider when evaluating if a particular strategy (in this case, the treatment-policy strategy) accurately represents the clinical question being investigated. - Promote a panel and audience discussion: The intention is to spark a deeper conversation on the predominance of the treatment-policy strategy and whether it truly reflects what clinical researchers want to learn about new treatments or medicines. - Encourage interdisciplinary involvement: The slide seeks to advocate for involving different disciplines in the estimand conversation, acknowledging that clinical trials are complex and may require input from multiple fields. - Clarify the purpose of clinical trials: A broader reflection is posed about the purpose of clinical trials, asking the question: What do we truly want to learn about new medicines?

Key Discussion Points:

1. Default to Treatment-Policy Strategy:

   • Explanation of treatment-policy strategy: In this strategy, the occurrence of intercurrent events (such as discontinuation of treatment or use of additional therapies) is considered irrelevant in defining the treatment effect. This approach is often based on statistical rationale rather than clinical reasoning.

   • Regulatory context: Regulatory advice often defaults to this strategy, where outcomes are analyzed as though the treatment continues, even after an intercurrent event. This can sometimes overlook the clinical context, focusing purely on statistical considerations.

   • Introduction to the Issue: The slide begins by presenting the issue that regulatory guidance often defaults to using the treatment-policy strategy. This strategy essentially assumes that certain events (such as discontinuation of treatment) are irrelevant, focusing more on a generalized statistical approach rather than the clinical reality of the situation.

   • Example to Illustrate: An example is provided (though not fully detailed in the explanation), involving a drug named Mylotarg for AML (Acute Myeloid Leukemia). The aim of this example is to show how a treatment-policy strategy might fail to isolate the impact of survival but still suggest the drug is beneficial based on broader statistical results. The slide prompts the audience to think about whether this approach truly answers the clinical question or whether more nuanced strategies would provide clearer insights.

2. Promote Panel and Audience Engagement:
   • The slide proposes two critical questions for discussion:
     • Why is the treatment-policy strategy so prevalent? Is this the best way to answer clinical questions, or does it simply reflect convenience in data handling?
     • When does the strategy truly align with the clinical question? Does it always serve the purpose of what the clinical trial intends to uncover about a drug or treatment’s efficacy?
3. Involving Other Disciplines:
   • The importance of bringing in different disciplines—such as statistics, clinical experts, and patient advocates—into the discussion is emphasized. This can enrich the conversation and help ensure that the trial design and strategies used are both clinically relevant and statistically valid.
4. Reflection on Clinical Trials’ Purpose:
   • The broader purpose of clinical trials is questioned: What are we really trying to learn? Are we trying to learn about a drug’s effect in real-world settings (which might be aligned with a treatment-policy strategy), or are we more interested in understanding the precise biological effect of the drug itself, separated from intercurrent events?

To Do with Slide

1. Changing Control Arm and Treatment Sequences:
   • One of the core issues highlighted is the evolving nature of control treatments in oncology trials. Often, new experimental treatments are introduced as standard of care evolves, which complicates the interpretation of results over time.
   • For example, a new drug might initially be tested as a third-line treatment (after failure of other treatments) and then later moved up to being tested as a second-line treatment. This change in treatment lines could lead to comparisons between different treatment policies (strategies) at different points in time, which becomes difficult to interpret.
2. Crossover in Treatment Strategies:
   • In oncology trials, there are cases where participants in the control arm may crossover to the experimental treatment. This crossover introduces a comparison between two different treatment sequences, which complicates the estimation of the treatment effect.
   • For instance, if a patient is initially on a control treatment but later switches to the experimental treatment due to progression, the trial might end up comparing the control treatment sequence with the experimental treatment sequence. This kind of crossover is particularly relevant in cases where a drug is being tested in later treatment lines (e.g., second or third-line therapies).
3. Defining Treatment Strategies:
   • Defining and maintaining a consistent treatment strategy throughout the trial is critical. However, as standard of care changes over time, the control arm in a trial may evolve, making it difficult to stick with the originally defined treatment strategy.
   • The trial’s estimand (the quantity being estimated, such as treatment effect) might not hold by the time the trial concludes, especially if the control strategy has changed. This introduces complexity in interpreting the final results because the comparison may not represent a consistent clinical question over the course of the trial.
4. Challenges in Estimating Efficacy and Safety:
   • As the control arm evolves during the trial, the estimation of treatment effects becomes complicated. For example, if a standard of care treatment changes halfway through the trial (due to new drugs being introduced), comparing a new treatment to a dynamic control arm could lead to estimates that are difficult to interpret. The comparison will only be meaningful if the changes in the control arm are fully understood and accounted for.
   • This is especially important in long-term trials where the standard of care may evolve significantly over time. The trial may start with one control treatment and end with a completely different one, making the treatment policy estimand less reliable.
5. Regulatory Consistency and Treatment Policy:
   • The discussion raises the question of regulatory consistency. Should regulatory bodies consistently use the treatment-policy strategy, even when it leads to inconsistencies in comparisons? The treatment-policy strategy assumes that any intercurrent event (such as a patient switching treatments) is irrelevant to the analysis of treatment effect, but this assumption might not hold in rapidly changing therapeutic areas like oncology.
   • Regulatory consistency might require maintaining the treatment-policy strategy across all trials, but this approach could lead to inconsistencies in how comparisons are made and how treatment effects are estimated over time. Regulatory bodies need to balance the need for consistency with the evolving nature of clinical practice.
6. Impact of New Drugs on Treatment Policy Estimates:
   • As more new drugs are introduced into the standard of care, trials may be affected by the increasing use of these treatments in the control arms. For instance, in trials testing anti-amyloid drugs for Alzheimer’s disease, if three drugs are introduced sequentially over time, the third trial (run after the first two) might have a more effective control arm due to the cumulative addition of new therapies.
   • This can result in smaller treatment policy estimates over time, as the control arm becomes stronger due to the evolving standard of care. The first trial might show a larger treatment effect than the third, even if all the drugs are equally effective, simply because the control arm has changed.

4. TP Question Complicated by Rescue Medication

The discussion around treatment-policy questions complicated by rescue medication deals with how rescue treatments (administered during a trial to manage symptoms) can affect the interpretation of trial results, especially when evaluating long-term outcomes and comparing the effects of an experimental drug against a control or placebo.

• Rescue medication introduces complexity in trials, particularly when using a treatment-policy approach that considers the entire treatment sequence. The use of rescue medication can obscure the true effect of the experimental drug.
• To avoid this issue, it may be more informative to adopt a hypothetical strategy that isolates the drug’s effect by considering what would happen without rescue medication.
• This approach would provide more direct information to prescribers and regulators, enabling better decisions about the drug’s efficacy and safety.

1. Challenges in Assessing Treatment Effects with Rescue Medication:

• The treatment-policy question aims to assess the overall effect of a treatment, including the effects of any subsequent treatments, like rescue medication. This can lead to complications when analyzing the impact of a drug, especially when rescue medications are used frequently.
• In trials focused on long-term outcomes, it can be harder to isolate the effect of the experimental drug because outcomes may be impacted by a sequence of treatments, including rescue medications.

2. Scenario A – Treatment-Policy Strategy and Rescue Medication:

• Scenario A introduces a situation where the experimental treatment is poorly tolerated, and a large proportion of patients switch to a rescue medication.
• Patients in the experimental arm switch to effective rescue treatment (50%) because of poor tolerability, while a smaller proportion (10%) in the control arm switch to rescue therapy.
• Despite showing superiority in the treatment-policy question, the result doesn’t seem helpful for making regulatory decisions. This is because the comparison ends up being between treatment sequences rather than a clear assessment of Drug Z’s effect alone.
• As a result, the estimate may be uninformative for regulators or prescribers, as it reflects the effectiveness of the entire treatment sequence (including rescue medication) rather than the standalone effect of the experimental drug.

3. Scenario B – Interpretability to Prescribers:

• Scenario B questions the interpretability of estimates when rescue medication use is prevalent.
• The estimate only makes sense if prescribers understand the rules governing rescue medication, including how and when it is used and the number of patients switching to rescue therapy.
• Different protocols with varying rules for rescue medication will yield different treatment effect estimates, which can complicate the assessment of Drug Z’s efficacy. In other words, the estimate obtained in one trial may not be generalizable to clinical practice, as it could be dependent on the specific rescue medication rules used in the trial.

4. Treatment-Policy Questions and Hypothetical Strategies:

• The discussion raises the idea of asking a hypothetical question to isolate the effect of the experimental drug from rescue medication. If the treatment-policy estimate reflects the entire sequence, including rescue, then perhaps we should instead ask, “What would the treatment effect be without rescue medication?”
• This shift in questioning allows for a hypothetical estimand that isolates the effect of Drug Z from the influence of rescue medications. While treatment-policy questions are seen as more conservative, they may not always reflect the true treatment effect of interest when rescue medications are involved.

5. Conservative vs. Hypothetical Approach:

• The treatment-policy approach is often described as conservative because it includes the effects of rescue medications, which can dilute the observable effect size of the experimental drug. The control arm may benefit from effective rescue medications, making the drug’s advantage appear smaller.
• If the goal is to understand the drug’s direct effect, the hypothetical strategy may be more appropriate, as it asks what the treatment effect would be without rescue medication. This approach avoids complications from rescue medications and may provide clearer insights for regulatory approval and clinical decision-making.

6. Communication to Prescribers:

• The slides emphasize that for the treatment effect to be interpretable to prescribers, they need to understand how the rescue medication was used and how it influenced the trial’s outcomes. Otherwise, the resulting estimate can be misleading, especially if it includes the effect of rescue treatments not typically used in real-world clinical practice.

4.2 S2-Talk 2: Clinical Perspectives on Estimand Framework Implementation.

Speaker: Miya Okada Paterniti (FDA, US, virtual)

Outline • Benefits of productive interdisciplinary collaboration • Challenges of implementing the estimand framework and discussing the clinical question of interest • Impact of estimand framework on regulatory processes

Engagement Between Clinicians and Statisticians

• The estimand framework aims to describe the treatment effect of interest in a way that aligns with the clinical question being addressed.

• Clinicians play a vital role by ensuring that the estimand captures the relevant clinical effects and takes into account various patient experiences or “journeys” during the trial.

• The goal is to incorporate intercurrent events, such as discontinuation of treatment or rescue medications, which could affect the assessment of the treatment effect. This ensures the estimand mirrors the actual clinical question and treatment objectives.

• The overlapping diagram illustrates a shift from estimands being distinct from clinical questions to their integration, where the estimand fully reflects the clinical question.

• Effective engagement between clinicians and statisticians is crucial to translate clinical questions into estimands.

• Clinicians provide the clinical insights to formulate the appropriate questions of interest, while statisticians ensure these questions are measurable and well-defined in statistical terms.

• Statisticians also define the estimand terminology, which lays the foundation for understanding the core objectives of the study, helping clinicians make sense of how statistical strategies are used to answer clinical questions.

• This collaboration helps ensure that trial designs and analyses align with the intended outcomes and clinical relevance.

Challenges in Estimand

• One of the biggest challenges in the estimand framework is the knowledge gap in terminology, particularly on the clinician side. Clinicians need to understand statistical terms, while statisticians need to grasp the clinical context.

• Each therapeutic area presents unique challenges, requiring tailored approaches to handle intercurrent events (e.g., patient dropout, treatment changes).

• A common struggle is implementing strategies (like composite outcomes) for continuous endpoints that account for negative outcomes.

• The “Learn as We Go” approach encourages trial designs that include supplementary analyses to assess the effects of intercurrent event strategies. These lessons can guide future trials in similar therapeutic areas.

• Clinicians are typically more familiar with certain strategies like the treatment-policy and composite strategies. Other strategies, like principal strata, are less familiar and require more explanation.

• It is helpful to relate these unfamiliar strategies to more commonly understood concepts like the intent-to-treat (ITT) principle. For example, explaining treatment policy strategies in terms of ITT helps clinicians better understand how the framework works.

• Another common issue is distinguishing between strategies for handling missing data and intercurrent event strategies (ICEs). This needs to be made clear early on in the discussion to prevent misunderstandings.

• Clinicians often respond better when ICE strategies are framed in clinical terms, so the communication process can be smoother when statistical terms are translated into clinical language.

Reframing Strategies into Clinical Questions

This approach is essential in engaging clinicians more effectively in discussions about estimands, as it ties statistical concepts back to clinical relevance.

1. Treatment Policy Strategy: - Clinical Question: “What is the effect of the test treatment compared to control, regardless of whether the intercurrent event occurred?” - Example: If treatment discontinuation occurs (which is an intercurrent event), would you still want to know the effect of the drug compared to control, regardless of whether the patient continued or discontinued the treatment? - Explanation: This approach helps determine the overall treatment effect, irrespective of whether patients had to discontinue the treatment. This could reflect a more general understanding of how the drug works in real-world practice, where some patients may discontinue treatment for various reasons.

2. Composite Strategy: - Clinical Question: “What is the effect of the test treatment compared to control, reflecting the unfavorable nature of the intercurrent event on the interpretation of the outcome?” - Example: If death was the intercurrent event, should it be reflected in the assessment? If most subjects died during the trial but the effect was only measured in survivors, would that accurately reflect the clinical question? - Explanation: This strategy accounts for how unfavorable intercurrent events (like death) affect the outcome. It aims to provide a more accurate understanding of how the drug impacts survival or progression, even if the clinical situation worsens significantly.

Why This Matters:

• Clinician Engagement: By translating these strategies into clinical questions, the decision-making process becomes more intuitive for clinicians. It shifts the focus from abstract statistical strategies to real-world, patient-centric questions.
• Clearer Communication: Reframing strategies as clinical questions helps bridge the gap between statistical rigor and clinical interpretation, which is crucial for making informed decisions during drug development.
• Better Trial Design: This approach also encourages more thoughtful design of trials by ensuring that both clinicians and statisticians are aligned on how to handle events like treatment discontinuation or rescue medication within the trial’s statistical framework.

Key Challenges in Addressing Intercurrent Events

1. Context-Specific Assessment:
   • The importance of considering which intercurrent events are relevant to each clinical trial and patient journey is stressed. The intercurrent events may vary even within trials of the same disease. For instance, two trials evaluating different severities of a disease may handle intercurrent events differently.
   • Example: A first-line therapy trial may treat changes in background therapy as an intercurrent event, while an add-on therapy study may not.
2. Intercurrent Events Vary by Therapeutic Area:
   • Intercurrent events can differ significantly depending on the therapeutic area. For example, in some diseases, death may be a common outcome that must be addressed adequately in the estimand. Rescue medications are also common in certain therapeutic areas, particularly in diseases where symptom exacerbations require immediate treatment, which can impact conclusions about efficacy.
3. Complexity of Handling Intercurrent Events:
   • Some intercurrent events, such as treatment discontinuation, can have multiple causes (e.g., lack of efficacy vs. adverse events). The specific reason behind discontinuation may impact the efficacy conclusions differently.
   • There is also a risk of overlapping intercurrent events, where a patient may experience multiple events simultaneously. For instance, a patient may discontinue treatment due to lack of efficacy while also switching to a different treatment, further complicating the analysis.
4. Beyond Common Intercurrent Events:
   • In some diseases, unique intercurrent events like corrective surgery may need to be considered because they could affect treatment efficacy and outcomes. These disease-specific events require a thorough understanding of the patient journey and collaboration with clinicians to adjust for their impact on efficacy.

Strategies for Addressing Intercurrent Events

1. Exploring Different Strategies:
   • Selecting the appropriate strategy for each intercurrent event can be challenging. Some strategies, such as treatment policy and composite strategies, might be favored, but they can be complicated by issues like missing data or uncertainty regarding the hypothetical situation.
2. Supplementary Analyses:
   • Given the challenges in defining the best strategy for each intercurrent event, conducting supplementary analyses to explore the effects of different strategies can be helpful. This can offer additional knowledge that may inform future clinical trials and improve decision-making regarding intercurrent events.

4.3 S2-Talk 3: Implementation of the estimand framework in the regulatory assessment: How it started and how it’s going.

Speaker: Laura Rodwell (Medicines Evaluation Board, NL)

1. Role of Assessors at the Medicines Evaluation Board:

• The Medicines Evaluation Board (MEB) is part of the centralized regulatory process, particularly in the Netherlands. In this system:
  • Clinical assessors are assigned to evaluate a drug’s efficacy and safety. Usually, two clinical assessors are involved—one focused on efficacy and the other on safety.
  • Statistical assessors also contribute, primarily focusing on the clinical efficacy part, but they may step in to address other sections if necessary.
  • While clinical assessors typically lead the assessments, statistical assessors play a key role in areas where methodology is critical, particularly with the integration of the estimand framework.

2. Important Reports and Documents in the Process:

• The clinical assessment report is the most critical document produced during the evaluation. This report is created during the early stages of the assessment (e.g., up to day 80 in the assessment timeline).
• This first clinical assessment report includes a comprehensive evaluation of:
  • The study design.
  • Whether the estimand has been defined and whether it aligns with the clinical question of interest.
  • The results and their implications for the benefit-risk profile of the product.
• The initial evaluation results in an overview and list of questions that aim to resolve uncertainties, leading to a concise evaluation of the product’s key attributes.
• The process culminates in a CHMP (Committee for Medicinal Products for Human Use) opinion, after which the European Public Assessment Report (EPAR) is prepared. The Summary of Product Characteristics (SPC) is also developed during this process.

3. Challenges with the Estimand Framework:

• The estimand framework was introduced as part of the ICH E9 (R1) guidelines to address intercurrent events and ensure that clinical trials focus on answering the correct clinical questions.

• The statistical assessors were quicker to adopt the estimand framework, as it aligns well with their problem-solving approaches. However, clinical assessors had a steeper learning curve in integrating this framework into their assessments.

• The role of the statistical assessor evolved into that of a trainer and facilitator, helping the clinical assessors understand the estimand approach. There was a need to ensure that clinical assessors saw the estimand as a tool to resolve issues and ensure alignment between clinical and statistical assessments.

• The speaker highlights that while scientific advice was designed to discuss the estimand framework upfront, the integration of estimands into the assessment process was less clear and needed to be defined further as the process evolved.

• The estimand framework’s implementation required collaboration between clinical and statistical teams.

• The role of the statistical assessor has expanded to include educating clinical assessors on the estimand and ensuring its proper application in both the design and evaluation of clinical trials.

• Early and continuous discussion of the estimand during scientific advice meetings is crucial to aligning clinical trials with the correct clinical questions.

4.4 S2-Panel Discussion

Statement from (Greg Levin from the FDA)

Greg Levin from the FDA is emphasizing the need for clinical trials to be designed in a way that informs real-world treatment decisions for both prescribers and patients. He is advocating for aligning trial designs with actual clinical practice to improve their relevance and impact on public health.

• Greg’s approach advocates for a realignment of clinical trial designs to ensure they are more applicable to real-world decision-making.
• His emphasis on realistic control arms and careful use of hypothetical strategies suggests that trial outcomes should better mirror the experiences and decisions that patients and prescribers face in everyday healthcare settings.
• The design changes proposed could lead to trials that are more informative, reduce unnecessary complexities, and provide more reliable data for prescribers and patients when assessing new therapies.
• Greg’s points likely resonate with the ongoing effort to create clinical trials that are not only robust for regulatory approval but also practically relevant for healthcare providers, improving the adoption and impact of new treatments.

1. Core Principle Proposal:
   • Clinical trials should be designed to provide information that is directly relevant for informing prescribers and making real-world treatment decisions.
   • He suggests that removing the tension between trials designed for regulatory decisions and those meant to guide real-world practice can enhance the utility of trials for both purposes.
2. Control Arm Design:
   • He proposes that control arms in trials should represent current standard of care in the local context, especially in diseases with existing approved, safe, and effective therapies.
   • He criticizes trials that withhold standard therapies from participants in the control arm, particularly in placebo-controlled trials, which he says do not inform real-world treatment decisions effectively. He references example #4 from a previous presentation to illustrate these challenges.
   • His suggestion includes using more active control trials (superiority or non-inferiority designs) or add-on trials where therapies are tested alongside standard of care.
3. Hypothetical Strategies for Intercurrent Events:
   • He argues that hypothetical strategies should only be used in scenarios that reflect circumstances likely to occur in real-world settings.
   • He provides the example of COVID-19-related treatment interruptions in clinical trials, where hypothetical strategies were appropriate because they reflected real-world operational disruptions expected to resolve post-pandemic.
   • However, he suggests that using hypothetical strategies without real-world applicability can lead to less informative results for actual practice.

Statement from (John Johnston from MHRA, UK)

1. Implementation Metrics:
   • Estimands have been integrated into guidelines, EMA templates, and other official documents.
   • However, implementation is not yet fully realized, with some untapped potential.
2. Ownership of Estimands:
   • Currently, estimands seem to be largely owned by statisticians, while other stakeholders (like clinicians and regulatory colleagues) often see statisticians as the primary responsible party.
   • A future goal is to shift towards a shared governance model, where all relevant parties feel responsible for estimands, not just statisticians. In this model, clinicians and regulatory colleagues would take more ownership, while statisticians act as facilitators.
3. Estimator Level (Statistical Methods):
   • There has been significant focus on the conceptual level of estimands, which was necessary for understanding and teaching the framework. However, there is still a gap in developing and evaluating estimators (statistical methods).
   • While some work has been done on handling missing data and single intercurrent events (e.g., the work by Tobias mentioned), more needs to be done, especially when there are multiple intercurrent events with complex patterns.
   • Statisticians should lead the way in developing these methods, but they must work closely with clinicians to ensure that the assumptions and methods are clinically realistic and applicable to real-world scenarios.
4. Future Focus:
   • Moving forward, the aim is to further develop and evaluate estimators and ensure that clinicians and regulatory colleagues are more engaged in the estimand discussions, taking responsibility alongside statisticians.
   • The speaker envisions a co-ownership model where everyone involved in the clinical trial process plays an active role in defining and using estimands.
   • There’s a call to keep track of progress and see where things stand in the next few years.

Key Discussion Points

1. Language and Terminology in E9R1:
   • The new language introduced by the estimand framework (terms like “treatment policy” and “intercurrent events”) is creating barriers for teams unfamiliar with the concepts.
   • The speaker encourages plain language conversations with clinical teams, where the technical terms are avoided until the teams are comfortable with the concepts. This helps to foster a deeper understanding of the clinical questions without overwhelming teams with jargon.
   • The goal is to have clear communication to ensure everyone involved in the study understands the estimands without focusing on complex terminology too early in the process.
2. Practical Challenges in Neuroscience Trials:
   • The speaker references neuroscience trials, where events like patient deaths, treatment discontinuation, and rescue medication use are common. The estimand framework helps teams think about these events in advance and align on what clinical questions they want to answer.
   • There is an emphasis on planning for intercurrent events and having open discussions on how they should be managed to accurately reflect the clinical question of interest.
3. Consistency vs. Adaptability in Clinical Trials:
   • Consistency in trial designs and estimands is a recurring topic. While clinicians may strive for consistency by comparing trials to previous procedures, the speaker suggests that sticking with a past decision just for consistency’s sake may not be the best approach.
   • The example of Alzheimer’s treatment is discussed, where the changing standard of care over time can impact the estimated treatment effect. In such cases, the hypothetical strategy may give a more stable estimate over time compared to a treatment policy approach.
   • The key takeaway here is that consistency should not be about repeating the same decisions across trials, but rather ensuring that clinical questions and estimands are adapted to reflect the current context and real-world conditions.
4. Impact of Hypothetical Strategy:
   • The hypothetical strategy is suggested as a potentially more consistent approach in cases where the control arm treatment sequence is changing, such as with the introduction of rescue medications with different rules or frequencies.
   • The speaker suggests that hypothetical strategies could provide more stable and reliable information over time compared to treatment policy strategies, especially when dealing with a shifting standard of care.
5. Alignment with Regulatory Guidelines:
   • The discussion touches on how existing regulatory guidelines (such as those for Alzheimer’s treatments) recommend a hypothetical strategy for symptomatic medication initiation.
   • The challenge arises when balancing the real-world application of treatment with regulatory standards—sometimes the regulatory recommendations may seem to violate the core principles proposed earlier, particularly when long-term trials are required for disease-modifying treatments but are less practical for short-term outcomes.
6. Looking Forward:
   • The panel is actively reflecting on ways to make estimands more relevant and practical for real-world use, with a focus on creating consistent trial designs that also adapt to changes in treatment standards.
   • The idea of developing a set of core principles for implementing estimands in clinical trials is proposed as a way to bridge gaps and guide discussions across different therapeutic areas.
7. Risk Aversion and Regulatory Challenges:
   • One participant raised the issue of risk aversion within the development and regulatory decision pipeline. Even when a development team is confident in their approach, uncertainty arises when dealing with multiple regulatory agencies. Different agencies may have varying expectations, leading to challenges in consolidating approaches and avoiding delays.
   • This risk aversion can also impact decisions on analysis methods. If multiple analysis methods are used, and some provide favorable results while others do not, the regulatory agency may reject the drug based on the less favorable analysis.
   • The preference for simpler settings, such as using treatment policy, is noted because it minimizes discussion and the need for extensive sensitivity analyses, thus reducing perceived regulatory risk.
8. Treatment Policy Strategy:
   • Treatment policy is often seen as the default strategy for intercurrent events. However, the panelists argue that it shouldn’t automatically be the default without carefully considering the specific clinical setting.
   • They point out that EMA guidelines do discuss other strategies and that, while treatment policy is frequently recommended, alternative strategies may be more appropriate depending on the clinical question.
   • The discussion emphasizes the need for a more nuanced conversation about which strategies are appropriate for each trial, rather than defaulting to treatment policy.
9. Intention-to-Treat (ITT) vs. Estimand Process:
   • One panelist reflects on the long-standing preference for Intention-to-Treat (ITT) analysis because it’s based on randomization, which ensures a fair comparison between the trial arms.
   • The panelist argues that moving away from ITT can degrade the fairness of the comparison, creating potential conflicts when regulators request ITT analysis while also expecting an estimand framework.
   • The speaker reassures the group that this conflict can be resolved through clear justification. When interacting with regulators, it is critical to clarify and justify the chosen approach, whether ITT or estimand, to ensure alignment with regulatory expectations.

5 Session 3: Regulatory landscape in China

Chairs: Kit Roes (Chair of MWP EMA, NL) and Emmanuel Zuber (Independent consultant, CH)

5.1 S3-Talk 1: Opportunities and Challenges in Clinical Research under China’s Scientific Regulatory System: Focusing on Innovative Drug Development.

Speaker: Dr Duanduan Cong (Center for Drug Development-CDE NMPA, CN)

Outline • Latest advancements in scientific regulatory system • Navigating Challenges and Seizing Opportunities in a Changing Landscape • Conclusion and outlook

NMPA and CDE

NMPA, or National Medical Products Administration, is the primary regulatory authority in China responsible for the oversight of drugs, medical devices, and cosmetics. Its major function is to ensure the safety, efficacy, and quality of these products.

CDE is a key affiliated agency of the NMPA that is primarily responsible for the scientific evaluation and decision-making regarding drug applications. The main responsibilities of CDE are highlighted in the image and include:

1. Drugs, Vaccines, and Advanced Therapeutic Products: Overseeing the regulation and approval processes for these items.
2. Technical Evaluation: This involves the review of Investigational New Drug applications (INDs), New Drug Applications (NDAs), Abbreviated New Drug Applications (ANDAs), and major changes to existing approvals.
3. Formulation of Specifications and Technical Guidelines: Developing standards and guidelines to assist in the drug evaluation process.
4. ICH-Related Work: Engaging in activities related to the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use, which aims to standardize and improve the drug approval process internationally.

Organizational Structure

The organizational chart in the image is divided into three main categories:

1. Administrative Divisions: Likely involved in the overall governance, policy-making, and administration within CDE.
2. Evaluation Divisions: Focused on the assessment and review of drug applications.
3. Evaluation Support Divisions: These divisions probably provide technical and logistical support to the evaluation processes.

Transition in Drug R&D

The transformation of China’s pharmaceutical industry and regulatory landscape reflects a strategic shift from primarily focusing on generic drugs to a balanced development of both innovative and generic medicines. This adjustment aligns with a broader commitment to enhance domestic capabilities in drug research and development (R&D) and to cater to global competitive pressures.

1. Shift in Focus: Since the reform in 2015, China’s drug R&D has transitioned from emphasizing generic drug production to fostering an environment that equally values innovation and imitation. This change is propelled by the need to compete globally and to meet internal demands for more advanced therapeutic solutions.

2. Growth in Drug Trials: The number of new drug clinical trials has seen a year-on-year increase, indicating a robust pipeline for innovative drugs. For example, from 2020 to 2023, there was a steady rise in the registration of new drug trials, showcasing heightened activity in drug development:

   • 1,473 trials in 2020
   • 1,933 trials in 2021
   • 1,974 trials in 2022
   • 2,323 trials in 2023

   The largest proportion of these trials has been in the field of oncology, exceeding 40%, highlighting a significant focus on cancer therapies.

3. Diversity in R&D: The R&D pipeline is varied, targeting diseases across different systems such as endocrine, digestive, and mental health, along with critical areas like blood disorders and anticancer therapies. This diversity underpins the potential for launching multiple innovative drugs in the future.

Regulatory Enhancements

1. Scientific and Rational Framework: Ongoing refinements to drug regulatory laws aim to align more closely with global standards, ensuring a scientific and rational approach that maintains safety and efficacy throughout the drug development lifecycle.

2. Efficiency in Drug Approval: Reforms in the drug review and approval process have streamlined procedures, significantly reducing the time to market for new drugs while ensuring high safety standards. This not only accelerates patient access to new treatments but also improves the overall efficiency of the pharmaceutical sector.

3. Specialized Drug Categories: There is an increased focus on developing regulatory requirements for drugs targeting major diseases and special categories such as pediatric medications and narcotics. This ensures thorough review processes and enhances the safety and effectiveness of these drugs in clinical use.

4. Advanced Risk Management: The enhanced monitoring of drugs pre- and post-market includes a robust system for reporting adverse drug reactions, ensuring ongoing safety assessments throughout the drug’s lifecycle.

5. Digital Advancements: The adoption of digital tools and the development of national drug supervision platforms have enhanced the transparency and efficiency of regulatory processes, facilitating real-time data sharing and more effective oversight.

6. Focus on Clinical Value and Public Engagement: Drug regulations increasingly prioritize clinical value and public involvement, ensuring that the development and approval of new drugs meet actual clinical needs and engage public feedback, thereby aligning drug development with patient-centric outcomes.

Revamping Drug Regulation

1. Strengthening the Legal Framework:
   • Objective: To improve legal provisions concerning the entire lifecycle of a drug.
   • Actions: This involves enhancing pre-market drug review and approval processes, supervising drug production and distribution, and monitoring adverse reactions to drugs.
2. Improving the Drug Review and Approval Process:
   • Objective: To implement new policies and measures that ensure efficient drug approval.
   • Actions: Adoption of a nationwide marketing authorization holder system, implementation of clinical trial implicit approval and bioequivalence notification systems, and setting up frameworks to support clinically valuable drug innovations.
3. Refining the Regulatory Requirements for Key Areas and Special Drugs:
   • Objective: To create more detailed regulations tailored to specific types of drugs or patient populations.
   • Actions: Formulation of regulations for oncology drugs, drugs for rare diseases, therapies for special populations, drugs for advanced diagnostics, and therapeutic drugs.
4. Enhancing Risk Management:
   • Objective: To manage risks throughout the drug lifecycle effectively.
   • Actions: Regulation of pharmacovigilance activities, which involves monitoring the effects of drugs after they have been approved for use to ensure they remain safe and effective.
5. Leveraging Technology for Better Oversight:
   • Objective: To enhance the efficiency and transparency of drug regulation.
   • Actions: Establishment of a national drug electronic supervision platform to track the complete process of drug production, circulation, and usage.
6. Clinical Value Oriented and Public Engagement:
   • Objective: To prioritize clinical value and involve the public in the drug development process.
   • Actions: Ensuring that the development and evaluation of new drugs are clinically oriented and that patient perspectives are included, particularly in the development of innovative treatments.

Overall, these steps aim to modernize and make more effective the regulatory framework surrounding the development, approval, and monitoring of pharmaceuticals in response to evolving healthcare needs and technological advancements.

Accelerating Market Launch to Support Clinical-Value-Oriented Drug Innovation

1. Special Review and Approval Procedure: - Activated during public health emergencies. - Facilitates quick approval of essential therapeutic and prophylactic drugs as mandated by law.

2. Priority Review and Approval Procedure: - Targets drugs with urgent clinical needs or those addressing major infectious diseases and rare diseases. - Includes innovative drugs, pediatric formulations, vaccines, and breakthrough therapy drugs. - Emphasizes drugs that can significantly improve clinical outcomes.

3. Breakthrough Therapy Designation (BTD): - Applies to drugs for serious life-threatening conditions that significantly improve upon existing therapies. - Requires strong evidence demonstrating substantial clinical benefits over existing treatments.

4. Conditional Approval Procedure: - Used for drugs treating serious diseases with no effective treatments. - Based on promising clinical trial data predicting the drug’s value. - Includes vaccines needed urgently for public health emergencies.

These procedures are designed to expedite the development, review, and approval of drugs that address critical and urgent health needs, enhancing the speed and responsiveness of the healthcare system to emerging challenges.

Evolving Regulatory System in Alignment with ICH

1. International Collaboration: - NMPA’s participation in the International Council for Harmonisation (ICH) since 2017. - Active involvement in setting global standards, having adopted all 69 ICH guidelines.

2. Implementation of ICH Guidelines: - Fast-tracked through targeted training and the publication of interpretative documents. - Enhances integration into drug R&D and approval processes.

3. Global Influence and Collaboration: - NMPA has strengthened global ties and influence in pharmaceutical regulations through active participation in ICH activities. - Focus on aligning domestic regulations with international standards to improve the global competitiveness of China’s pharmaceutical industry.

These efforts by the NMPA to align with ICH standards and expedite drug approvals underscore a commitment to integrating global best practices into China’s regulatory framework. This strategy not only improves the efficiency and efficacy of the drug approval process but also enhances China’s role in the global pharmaceutical landscape. These initiatives ensure that innovative treatments, particularly those with high clinical value, reach the market more quickly, benefiting patients both domestically and globally.

Rising Tide, Rising Issues: Innovative Drug Development in a Changing Landscape

• The necessity for regulatory evolution and international collaboration is highlighted as being pivotal for accommodating the rapid pace of innovation within the pharmaceutical industry.
• China’s unique position in the global market as a growing hub for pharmaceutical R&D is underscored by its commitment to regulatory reforms and strategic initiatives that support a robust environment for drug innovation.
• The push for global synchronization in drug approval processes is not just beneficial for market dynamics but crucial for patient access to timely and innovative treatment options globally.

Provides a clear overview of the evolving landscape of innovative drug development, emphasizing the need for proactive strategies to stay ahead in a dynamic pharmaceutical sector.

1. A Pressing Need for Groundbreaking Innovation:
   • Focus: Original innovation remains the cornerstone of drug development, which demands an increase in basic research funding and encouragement for original thinking to avoid homogeneity in product offerings.
   • Challenges: While there has been progress, the pipeline for truly groundbreaking drugs does not fully meet potential. To bridge this gap, fostering a culture that supports pioneering research and development is crucial.
2. Elevating the Foresight of Technological Standards:
   • Importance: As new technologies and methodologies emerge, it’s essential to establish and maintain a dynamic and forward-looking system of technological standards.
   • Approach: Regulatory standards must evolve continuously to accommodate and encourage cutting-edge developments, ensuring that the pharmaceutical industry can effectively integrate innovations and enhance the quality and speed of drug development.
3. Strengthening International Collaboration for Global Impact:
   • Objective: Enhance global cooperation to align technical requirements and regulatory standards, promoting harmonized approaches to drug approval.
   • Benefits: This effort will ensure that innovative drugs receive simultaneous approval across different regions, reducing time to market and providing patients worldwide quicker access to new therapies.

Conclusion and Outlook: Advancing Pharmaceutical Innovation and Global Health

As China continues to develop its innovation ecosystem, marked by a growing number of high-tech startups and research initiatives, it further solidifies its position as a global hub for pharmaceutical research and development. Despite facing several challenges, the convergence of these dynamic factors provides a fertile environment for groundbreaking advancements in drug development. By leveraging its strengths and strategically addressing remaining hurdles, China is poised to make significant contributions to global health, offering new solutions for pressing clinical needs.

To effectively advance the field of innovative drug development and meet the evolving demands of healthcare: - Aligning Standards with Global Practices: It’s critical to align our drug review standards with international best practices. Active participation in global regulatory harmonization efforts will streamline processes and facilitate the development of new therapies worldwide. - Accelerating Marketing of Urgent Drugs: Continuing to deepen reforms in our review and approval systems is essential. This will speed up the availability of clinically urgent drugs, particularly those targeting severe life-threatening conditions. - Prioritizing Oversight and Quality: We must prioritize the oversight of innovative drugs and advanced therapies, focusing intensively on drug quality, efficacy, and safety to ensure public health and that new treatments meet the highest standards. - Supporting Global R&D and Special Populations: Encouraging simultaneous global R&D efforts and fostering the development of drugs for special populations, such as those with rare diseases and pediatric conditions, will broaden the scope and impact of our pharmaceutical innovations. - Enhancing Regulatory Science and Supervision: Committing to regulatory science research and exploring innovative supervision methodologies will enhance the efficiency and intelligence of drug oversight. - Strengthening International Collaboration: Strengthening relationships with regulatory agencies worldwide and fostering collaboration among government, academia, and industry is crucial for promoting innovation and supporting the development of new drugs. - Building a Diverse Talent Ecosystem: Creating a strong talent base and a diverse and dynamic ecosystem for drug innovation will further propel our progress.

5.2 S3-Talk 2: Implementation of ICH Statistical Guidelines in China: from the Regulatory Perspective.

Speaker: Dr Jianhong Pan (CDE, CN)

• Estimand doesn’t exsit in SAP and Protocol
• Recent evolutions in clinical trial design, like innovative design and analysis methods (e.g., estimand), highlight ongoing challenges in MRCT evaluations, suggesting that the E17 guideline needs continuous updates or supplements to address these advanced topics.
• The pooling strategy must be pre-specified, helping to provide flexibility in sample size allocation and facilitating the assessment of consistency across diverse patient populations.
• While the pooling strategy offers certain benefits, it also has limitations and is not mandatory under E17 unless specifically suited to the trial’s design.

5.3 S3-Talk 3: Intelligent regulation and statistics promote the modern development of regulatory science in China.

Speaker: Prof Hou (Peking University, CN)

Outline

• Overview Of China’s growing focus on regulatory science, especially with AI.
• Statistical methods in modernizing regulatory processes.

Strengthening Regulatory Frameworks and AI and Advanced Technologies Integration

• The speaker emphasizes China’s commitment to regulatory science with the establishment of over 100 national labs and collaboration with around 15 academic institutions focusing on regulatory science. This infrastructure supports ongoing projects and innovation in regulatory methods, including AI and statistical methodologies.
• The integration of AI and advanced technologies into regulatory frameworks is designed to not only speed up the approval processes but also to ensure that these processes are more adaptable to new challenges and innovations.
• The focus on real-world data and evidence is indicative of a shift towards more pragmatic and patient-centered approaches in regulatory science, ensuring that therapeutic approvals are based on comprehensive data that mirror real-world scenarios.

Strengthening Regulatory Frameworks

1. Advancements in Regulatory Science:
   • China has significantly advanced its regulatory science landscape, with substantial improvements in the fields of pharmaceuticals, medical devices, and healthcare.
2. Adoption of International Practices:
   • The National Medical Products Administration (NMPA) is adopting international best practices to ensure the safety and efficacy of products. This adoption reflects a global alignment with established regulatory standards.
3. Science-Driven Frameworks:
   • Emphasis is placed on science-driven regulatory frameworks to support innovation and modernization, ensuring that regulatory decisions are based on robust scientific evidence and technological advancements.

AI and Advanced Technologies Integration

1. Core Technologies:
   • AI, machine learning (ML), and big data technologies are central to modernizing China’s regulatory processes.
2. Streamlining Drug Approval:
   • AI is employed to streamline drug approval processes, enhancing efficiency and effectiveness, particularly in handling complex data sets and managing post-market surveillance.
3. Real-World Data and Evidence:
   • The integration of real-world data (RWD) and real-world evidence (RWE) into the regulatory decision-making process marks a significant shift towards more adaptive and responsive regulatory practices. This integration aids in making informed decisions that are reflective of actual patient outcomes and experiences.

Key Aspects of Innovation-Driven Regulation

The slide titled “Focus on Innovation-Driven Regulation” highlights China’s strategic direction in regulatory science, specifically how it facilitates the integration of innovative therapies and medical devices.

1. Faster Approval Processes:
   • The shift towards innovation-driven regulatory frameworks is designed to allow for faster approval of novel therapies and medical devices. This approach reduces the time to market for groundbreaking treatments, ensuring that they can reach patients more swiftly.
2. Support for Emerging Therapies:
   • There is a significant emphasis on providing regulatory support for emerging and advanced therapeutic areas such as gene therapies, cell therapies, and personalized medicine. These areas represent the forefront of medical innovation and require regulatory frameworks that can accommodate their unique challenges and complexities.
3. Adaptation to Technological Advancements:
   • China is at the forefront of creating regulatory pathways that are responsive to technological advancements, particularly in biopharmaceuticals and digital health. This proactive approach ensures that the regulatory environment evolves in tandem with technological progress, facilitating innovation while maintaining safety and efficacy standards.

AI in Drug Discovery and Development

• Accelerating Processes: AI is significantly enhancing the efficiency of the drug discovery and development process. By utilizing machine learning models and algorithms, AI can analyze vast datasets to identify potential drug candidates faster than traditional methods. Additionally, AI can optimize clinical trial design and predict outcomes, thereby streamlining the development process and reducing time to market.

• Real-Time Monitoring: AI-powered systems are being employed to monitor adverse drug reactions (ADRs) in real time. These systems can process large volumes of data from clinical trials and real-world use, detecting patterns and signals that may indicate potential safety issues. This proactive surveillance helps in ensuring that drugs remain safe for the public post-approval, enhancing patient safety and compliance.

• AI-Driven Precision Medicine Supporting Personalized Medicine: AI plays a crucial role in the advancement of personalized medicine, where treatments are tailored to the individual characteristics of each patient. Through AI algorithms, it is possible to analyze genetic information, lifestyle data, and clinical histories to devise personalized treatment plans that are more effective and have fewer side effects. AI-driven diagnostics also contribute by identifying disease with greater accuracy and at earlier stages.

Overall Impact

• Enhanced Efficiency and Effectiveness: The integration of AI into regulatory science not only speeds up the drug approval process but also enhances the precision and effectiveness of the pharmaceutical industry’s ability to serve patient needs.
• Innovative Approaches: AI introduces novel methodologies in regulatory practices, allowing for more sophisticated analyses and decision-making processes based on comprehensive data analysis.
• Risk Management: By improving detection of ADRs and tailoring medication to individual needs, AI reduces the risk profiles of new drugs, thereby ensuring higher safety standards.

Strategic Challenges

Challenges and Solutions

1. Data Privacy and Security:
   • Significant attention is given to ensuring data privacy and security, especially important when integrating AI into sensitive areas such as healthcare and pharmaceuticals. Institutions like Peking University are actively working to address these concerns through dedicated research and development.
2. Talent and Infrastructure Development:
   • Recognizing the need for a skilled workforce and robust AI infrastructure, special training programs and infrastructure development initiatives are being implemented to support the evolving regulatory landscape.
3. Algorithmic Transparency:
   • To ensure that AI-driven decisions are fair, ethical, and transparent, guidance and laws are being developed. This transparency is crucial for maintaining trust in AI-assisted regulatory processes.
4. Methodological Innovations:
   • Adoption of advanced methodologies, like Model-Informed Drug Development (MIDD), is being used to predict clinical outcomes and optimize trial designs. This approach is part of a broader strategy to enhance the quality and effectiveness of regulatory reviews.

Data Privacy and Security

• Protection of Sensitive Data: One of the foremost challenges in using AI for regulatory decisions is ensuring the protection of data privacy and security. As AI systems often require access to vast amounts of sensitive data, it is crucial to implement robust data protection measures to prevent unauthorized access and data breaches.
• Compliance with Regulations: Regulatory bodies must also ensure that AI systems comply with international data protection laws and standards, such as the GDPR in Europe or HIPAA in the United States, to safeguard personal and medical information.

Talent and Infrastructure

• Skilled Workforce: The effective use of AI in regulatory science requires a workforce that is not only skilled in regulatory affairs but also proficient in AI and data science. There is a need to train existing employees or attract new talent with these hybrid skills to manage and interpret AI-driven systems effectively.
• Developing Infrastructure: Supporting the AI integration also demands advanced IT infrastructure capable of handling complex AI tasks, including data storage, processing, and analysis at a large scale. Investment in such infrastructure is necessary to harness the full potential of AI in regulatory processes.

Algorithmic Transparency

• Fair and Ethical Decisions: Ensuring that AI-driven decisions are fair, ethical, and transparent is another significant challenge. There is a risk of bias in AI algorithms, which can lead to unfair or discriminatory outcomes if not properly managed.
• Transparency and Accountability: It is important for regulatory bodies to develop guidelines that ensure AI systems are transparent in their operations and decision-making processes. This includes clear documentation of how AI models are built, trained, and deployed, as well as mechanisms for auditing and challenging AI decisions.

Research and Implementation of AI and RWE

• Flexibility and Cost Reduction: By adopting new scientific designs and utilizing AI, the goal is to increase flexibility in regulatory processes and reduce development costs. This approach leverages advanced analytical capabilities of AI to handle complex data efficiently, potentially streamlining regulatory pathways and reducing time-to-market for new therapies.

• Real-World Evidence Utilization: There’s a significant focus on integrating RWE into the regulatory framework. RWE involves data gathered from real-world medical settings—outside of conventional clinical trials—which can offer insights into how treatments perform in broader, more diverse populations. AI tools are employed to analyze this healthcare data to generate actionable insights that inform regulatory decisions, enhancing the relevance and applicability of regulatory assessments.

• ICH E17 Implementations and Workshops: The integration of the ICH E17 guidelines into China’s regulatory framework has been a focal point of recent efforts. Workshops that bring together academic experts, industry professionals, and regulators have been crucial. These workshops discuss the implementation challenges and solutions regarding the guidelines, facilitating a collaborative approach to regulatory innovation.

• Academic and Industry Collaboration: Experts from top universities and biotech companies are engaged to develop robust statistical frameworks and ensure practical, viable development models. This collaboration helps align academic research with industrial practices and regulatory standards.

• Development of Industry Blue Books and Consensus Reports: Outputs from these collaborations include industry blue books and consensus reports, which aim to guide the pharmaceutical industry and regulatory bodies. These documents compile shared insights and provide comprehensive guidelines for effectively navigating the complexities introduced by the new regulatory frameworks like ICH E17.

• Addressing Implementation Issues: The discussions and collaborations aim to address and resolve practical issues encountered with the implementation of ICH E17 in China. By identifying these challenges and working through consensus-building activities, the goal is to create adaptable solutions that can be integrated into the regulatory practices.

ICH E17 Guidelines Implementation and Focus Areas

1. Consistency Evaluations and Strategy Development:
   • The focus is on ensuring consistency in evaluations across various regions and countries involved in MRCTs. This is crucial for harmonizing drug approval processes globally and ensuring that drugs are evaluated fairly and thoroughly based on uniform standards.
2. Design and Planning Stages:
   • During these stages, careful consideration is given to disease diagnostics, treatment variations across populations, and racial factors that may affect drug efficacy and safety. The planning also includes strategic decisions on combined strategies and regional sample size allocations, which are essential for the logistical success of MRCTs.
3. Framework Development:
   • A robust framework is being developed to standardize approaches and ensure all stakeholders—including regulators and industry participants—are on the same page. This includes detailed guidelines on how to approach the trial design, including the assessment of efficacy and safety and the evaluation of benefit-risk balances.

Special Considerations and Statistical Methods

• Addressing Unique Challenges: The approach takes into account special considerations such as non-majority trials, multiple primary endpoints, interim analyses, delayed effects, and adaptive designs. These considerations are crucial for trials involving complex diseases or treatments that require flexible and adaptive trial designs.

• Statistical and Methodological Development: There is a focus on developing statistical methods that can handle the complexities of MRCTs, especially in understanding and addressing inconsistencies across different regions. This also involves predicting long-term outcomes based on early trial data, which is particularly challenging in diseases like gastrointestinal (GI) cancers prevalent in China.

1. Predicting Long-Term Endpoints:
   • The use of surrogate endpoints, such as Objective Response Rate (ORR) and Progression-Free Survival (PFS), to predict Overall Survival (OS) in GI cancer trials is highlighted. The effectiveness of these surrogate markers varies significantly depending on the molecular characteristics of the population and the type of therapy used.
2. Challenges with Surrogate Endpoints:
   • In molecularly enriched populations, there is a strong correlation between PFS and OS, which can be useful for predicting long-term outcomes. However, ORR may vary dramatically based on therapy type, presenting challenges in using it as a reliable predictor of OS across different treatments.
3. Statistical Methodology – Bayesian Approach:
   • The Bayesian approach is particularly noted for its ability to incorporate historical data and prior information into current trials. This method allows for a more nuanced understanding of drug effects based on previously gathered data, enhancing the predictive accuracy of trial outcomes.
4. Borrowing Information:
   • Information borrowing in Bayesian statistics involves using data from similar studies to inform current trials, which can be particularly useful in cases where direct data on certain endpoints might be sparse or unclear. This includes borrowing effect sizes from similar drugs or utilizing survival curves from previous studies to inform the current analysis.

5.4 S3-Talk 4: Joint Efforts for Innovative Drug Development in China.

Speaker: Dr Xiaoni Liu (Novartis, CN)

Role in Regulatory Reforms and SH Guidelines Implementation

• Regulatory Reforms in China: She discusses the significant regulatory reforms that have taken place in China since 2015, which were aimed at enhancing the efficiency and transparency of the drug review and approval process. These reforms include the adoption of new laws and regulations that streamline the approval processes for clinical trials and new drugs.

• Adoption of ICH Guidelines: A pivotal aspect of these reforms was China joining the ICH (International Council for Harmonisation) in 2017, aligning China’s drug regulations with global standards. The implementation of ICH guidelines has been a key part of her discussions, illustrating how these guidelines have been integrated into the Chinese regulatory framework.

• SH E17 and MRCT Blue Book Project: She highlights her involvement in projects like the SH E17 MRCT (Multi-Regional Clinical Trials) Blue Book project team, which collaborates with various stakeholders to ensure effective implementation of these guidelines in China.

• Enhanced Drug Development and Approval Processes: The reforms have allowed for more rapid development and approval of new drugs in China, reducing the previous delays that could extend up to four or five years for bringing innovative drugs to the Chinese market.

• Simultaneous Development and Submission: The regulatory changes have facilitated simultaneous development and submission strategies, making MRCT the preferred approach and significantly shortening review timelines for drugs in China.

SH E17 MRCT Blue Book Project

The efforts to harmonize China’s regulatory practices with global standards through the SH E17 MRCT Blue Book project exemplify a comprehensive and collaborative approach. This initiative not only enhances the international compatibility of China’s drug approval processes but also ensures that these processes are scientifically rigorous and tailored to meet the specific needs of the Chinese population as well as global diversity in clinical practices. This strategic alignment is crucial for facilitating faster and more efficient drug approvals, ultimately leading to better patient outcomes.

• Project Stages:
  • The Blue Book project involves several stages including a comprehensive literature review, closed-door sessions for drafting and discussions, and a finalization phase where the collective insights are compiled into actionable strategies.
• Pooling Strategy:
  • The SH E17 guideline emphasizes the use of a pre-specified pooling strategy for regions or subpopulations based on established knowledge about similarities. This strategy helps in facilitating sample size allocation and consistency evaluation.
  • Participation in early studies or MRCTs is encouraged to gather relevant scientific knowledge which can inform the impact of ethnic factors on treatment effects.
• Sample Size Allocation and Consistency Evaluation:
  • There is no universal approach to sample size allocation in MRCTs. The guideline suggests maintaining a balance between proportional allocation and equal allocation.
  • Selecting interim criteria for consistency evaluation is crucial and may require multiple discussions among different functions and between authorities and sponsors. A holistic approach is recommended for these evaluations.

Not defined in the Blue Book, will be discussed in the next version